Degenerative skeletal disorders including osteoarthritis (OA) and osteoporosis (OP) are the result of attenuation of tissue regeneration and lead to painful conditions with limited treatment options. Preventative measures to limit the onset of OA and OP remain a significant unmet clinical need. MicroRNAs (miRNAs) are known to be involved in the differentiation of stem cells, and in combination with stem cell therapy could induce skeletal regeneration and potentially prevent OA and OP onset.The combination of stem cells and miRNA has been successful at regenerating the bone and cartilage in vivo. MiRNAs, including miR-146b known to be involved in chondrogenic differentiation, could provide innovative targets for stem cell-based therapy, for the repair of articular cartilage defects forestalling the onset of OA or in the generation of a stem cell-based therapy for OP.This review discusses the combination of skeletal stem cells (SSCs) and candidate miRNAs for application in a cell-based therapy approach for skeletal regenerative medicine. Degenerative skeletal disorders are specific diseases associated with the ageing bone. These disorders can be divided into two main diseases: osteoarthritis (OA) and osteoporosis (OP). Given OA and OP are the result of the loss of the regenerative capacity of the skeletal tissues, skeletal stem cells (SSCs) have been investigated with the aim of harnessing their potential to improve the symptoms and treatment of these pathologies.OA is a prevalent chronic disease and can be described as a heterogeneous condition, which results in joint signs and symptoms associated with defective integrity of the articular cartilage and changes to the bone at joint margins . ArticulArticular cartilage deterioration and onset of OA could potentially be prevented by repair of the initial articular cartilage defect. A number of research groups have looked to identify the presence of chondroprogenitor cells within the articular cartilage and in tissues directly surrounding the articular cartilage, such as the synovium , the groThe human skeleton reaches peak bone mass at around 30 years of age and, thereafter, bone mass is gradually lost. OP is a degenerative skeletal disorder, characterised by low bone mass and generalised disorder of the bone microarchitecture. OP is observed in men and women  and is a common cause of loss of bone mass and subsequent fracture . It is eCurrently, OP is treated with drugs which aim to increase bone density or inhibit bone resorption. Strategies include the use of bisphosphonates , 19, selStem cells have been shown to be regulated in part by microRNAs (miRNAs), which regulate genes involved with differentiation post-transcriptionally. MiRNAs are processed from longer primary transcripts which undergo processing in the nucleus and the cytoplasm to form small non-coding RNA, which average 22 nucleotides in length . SequencA stem cell is characterised by its ability to self-renew by means of asymmetrical cell division and its potential to differentiate into specialised types of cells, thereby retaining a pool of stem cells and simultaneously producing transit amplifying cells . Adult sThe term skeletal stem cell (SSC), preferred by the authors and used in this review in reference to our own data defines, specifically, a self-renewing stem cell that resides in postnatal bone marrow and can differentiate into cartilage, the bone, haematopoiesis-supportive stroma and marrow adipocytes. It is the SSC of the bone marrow stroma that is responsible for the regenerative capacity inherent to the bone.The heterogeneous population of cultured plastic adherent cells isolated from the bone marrow should be referred to as bone marrow stromal cells (BMSCs). However, it is acknowledged, the term mesenchymal stem cell (MSC), originally in reference to a hypothetical common progenitor of a wide range of “mesenchymal”  tissues, is commonly used and in this review will be retained where cited/used by others in the field.Additional to their differentiation and proliferative properties, SSCs have been proposed to possess immuno-modulatory properties which can regulate tumour evasion, autoimmunity and regulation of transplantation tolerance . A combiLoss of chondrocytes and diminishment of the surrounding specialised ECM is as a result of the inability for cartilage to undergo spontaneous endogenous regeneration. The use of cell-based therapies to repair articular cartilage defects aims to produce a fully functional joint surface, capable of tolerating stress and strain.Several studies have investigated the potential of SSCs in regenerating cartilage in animal models. For example, Im et al. induced osteochondral defects in to the patella grooves of rabbits, and autologous bone marrow-derived MSCs were applied to the defect sites. Histological and molecular analysis concluded that implantation of cultured MSCs could enhance cartilage repair . In expeA popular choice amongst research groups for investigating articular cartilage regeneration has been transplantation of SSCs combined with a scaffold. Previously, osteochondral progenitor cells expanded in vitro and dispersed into a type-1 collagen gel were transplanted into a full-thickness surgically induced articular cartilage defect in rabbits. At 24 weeks, the post-implantation subchondral bone was completely repaired with overlying articular cartilage . FurtherPrevious clinical studies have reported the therapeutic effect of MSCs administration in patients –48. NejaGiven OP is the result of altered bone remodelling, improving the efficiency or restoration of appropriate balance of this process would appear a natural strategy for the treatment of OP. It is known that SSCs can be induced to form osteoblasts when cultured on tissue culture plastic . HoweverThe use of bone tissue, autograft (patient derived) and allograft (donor), together with bone stem cells and progenitors has been examined. Marcacci et al., in a study of four patients with large bone defects, examined the potential of autologous culture-expanded SSCs onto a ceramic scaffold . No majoIn vitro models of stem cell differentiation have allowed for the analysis of miRNAs involved with post-transcriptional regulation of chondrocyte and cartilage development, as well as osteocyte and bone development. Such miRNAs are responsible for gene activation or suppression during the process of differentiation. A selection of miRNAs and their mRNA targets studied to date, known to be involved in stem cell differentiation, are listed in Table Given that miRNAs display the potential to regulate chondrogenic and osteogenic differentiation of stem cells, harnessing miRNAs offers an appealing strategy for skeletal tissue repair of cartilage or enhancement of differentiation of SSCs towards an osteogenic lineage for bone formation. The potential of miRNAs to augment articular cartilage regeneration has been demonstrated in a study conducted by Lolli and colleagues ••. MiR-2The potential of miRNAs to augment bone formation has been demonstrated in a number of murine studies. With both miR-138 and miR34a, a hydroxyapatite/tricalcium phosphate (HA/TCP) scaffold was utilised in order to localise stem cells subcutaneously. Chen et al. used a similar approach to study the role of miR-34a, which is a negative regulator of bone formation ••. hMSCsLi et al. used a miRNA intravenous therapy approach, without the use of the scaffold, to investigate the role of the positive regulator of osteogenic differentiation, miR-2861, on bone formation in mice . When anALP and OCN expression in photoactivated hADSCs compared to non-UV treated cells. In addition, the specific use of nanoparticle conjugated miR-148b resulted in delivery of miR-148b to the intracellular compartments of hADSCs, without the need for additional, potentially damaging, chemical-based methods of transfecting stem cells.For successful use of miRNA in stem cell therapeutics, it will be important to localise and minimise any miRNA off target effects. Qureshi et al. developed a technique for photoactivation of nanoparticle conjugated miR-148b •; miR-14The problems associated with degenerative skeletal disorders highlighted in this review indicate how miRNA could be used to treat these musculoskeletal conditions. The underlying aetiology of OA remains unknown which makes development of a treatment for this debilitating disease difficult. However, if initial chondral lesions can be targeted, the potential for a preventative approach in OA will become a clinical possibility. If the original chondral lesion can be repaired using stem cells enhanced to undergo chondrogenic differentiation efficiently with use of miRNAs modulation, inducing regeneration of the articular cartilage and reinstating integrity, then the degenerative changes, typical of OA could be reduced. Thus, an attractive approach, with knowledge of different miRNAs expression during chondrogenic differentiation, would be to administer specific miRNAs transfected stem cells to chondral defect sites to enhance articular cartilage regeneration capacity. The bone regeneration balance lost in osteoporosis can benefit from an SSC-based cell therapy which could potentially restore bone microarchitecture and composition to a healthy state. The approach of priming these SSCs with miRNA could lead to enhanced direction of SSCs towards osteogenic differentiation. MiRNAs have been shown to enhance bone formation in murine trials, and known mutations in miRNAs have been identified in human osteoporotic patients. This cell-based approach could be advantageous when applied at early stages of the disease in order to prevent further bone loss and minimise any potential fracture risk that can occur with disease progression. While consideration of miRNAs in skeletal disease therapy is still in its infancy, with considerable research still to be undertaken, the potential for the use of miRNA in a therapeutic context offers an exciting treatment option for a growing ageing population.


Morbidity and mortality in China are increasingly associated with lifestyle behaviors, e.g., smoking, poor nutritional choices, and physical inactivity. Lifestyle-related non-communicable diseases  are at critical levels globally, in turn their socioeconomic burdens. Knowledge of lifestyle-related health behaviors and beliefs of mainland Chinese would help inform the design and targeting of cost-effective health education for individuals and campaigns in the interests of promoting and protecting health, and preventing disease. This study's objective was to describe the lifestyle behaviors and beliefs of a sample of urban mainland Chinese, and their congruence with evidence-based guidelines for maximal health.A cross-sectional interview questionnaire study was conducted in which 835 mainland Chinese  from four urban areas participated.About half (52%) reported smoking to some degree with 21% being habitual smokers; 33% being above average weight; 62.1% met physical activity guidelines for health benefits; 92% being sedentary for 5.8 ± 3.40 h/d; 66% experiencing moderate/high stress; and sleeping 7.1 ± 1.31 h nightly with 35% reporting sleeping poorly. When standard serving sizes were considered, daily consumption of grains, fruits, and vegetables was reported to be consistent with dietary recommendations for good health, however, added salt (3.7 ± 7.42 tsp) and sugar (3.9 ± 12.99 tsp) exceeded recommendations. Life stress was rated moderate by 59.6% of respondents, with personal and family health stresses ranking highest . Regarding beliefs about importance of lifestyle behaviors to health, respondents' understanding was not consistent with evidence-based recommendations. Only 64% of participants believed smoking abstinence is highly important to health; 56% regular exercise; and 37% consumption of whole grains, 62% fruit and vegetables; and 54% maintaining a healthy body weight.To be congruent with established guidelines for healthy living, health promotion and disease prevention education for individuals and public campaigns warrants targeting health knowledge and beliefs of urban Chinese as well as lifestyle-related health behaviors. The roles of gender, education and living rurally, on lifestyle behaviors and beliefs of the Chinese, warrant elucidation. Such knowledge may help to design, target, and implement health promotion and disease prevention education programs for individuals and the general public in China, with a view to protect health, and reduce the prevalence of lifestyle-related NCDs and their risk factors, and their associated social and economic burdens.2.2.1.A cross-sectional descriptive study was designed based on a structured face-to-face interview questionnaire. The study was reviewed and approved through the ethics review process of the participating institutions. The participants provided signed informed consent prior to their data being included. The target population included adult residents in four urban areas in the Chinese provinces of Hubei, Liaoning, Hebei, and Shanxi. The valid sample size was 835. Of the total of 846, 11 respondents failed to give their ages or volunteers were under 18 years of age. Interviewers (n = 80) were qualified health professionals . At the time of this study, these individuals were students in masters of physical therapy programs. As part of their professional training related to practice competencies, these students were trained to interview people about their lifestyle-related health behaviors and beliefs. Standardized interviewing technique as a clinical competency helped ensure a standardized procedure was instituted including how to address questions. We selected a purposive snowballing sampling method in that student interviewers were asked to recruit at least 10 individuals for interviewing (range 10 to 15). These individuals could be on or off campus, with an emphasis on those they did not know personally. Student interviewers were instructed to inform all respondents not only about the nature of the questionnaire, but also that no identifying information was required. Interviews were conducted within two weeks. Each interview was 30 minutes in duration.2.2.A structured questionnaire was developed based on previous work conducted in Canada The questionnaire was pre-tested on a convenience sample (n = 7) and modifications were made to enhance clarity of the items. More unambiguous mutually exclusive choices were added for items with multiple response options. In the interest of cultural relevancy, the unit of consumption in the dietary section was set to grams. Two proficiently bilingual individuals independently translated the questionnaire into Mandarin. The questionnaire was tested for clarity and comprehension on a sample of four bilingual individuals and revised accordingly.2.3.p values were two-sided and alpha was set at 0.05. The data were analyzed using Statistical Package for Social Sciences .All variables were summarized using standard descriptive statistics including means, standard deviations, and frequencies. Normality of distribution of continuous variables was assessed using the Kolmogorov-Smirnov test. Chi-square tests compared the frequency of those categories of interest. Kendall's Tau-b Tests were performed to determine the association between ordered variables. Mann-Whitney U tests assessed gender differences for physical activity variables. The Kruskal-Wallis test for independent samples assessed level of significance for variables with non-normal distributions. Where statistical significance was observed for the Kruskal-Wallis test, the Mann-Whitney U test with Bonferroni corrections was used for post-hoc analysis. All 2We selected cut-points for recommendations for maximal health status that are established, evidence-based, and commonly reported 3.3.1.2; 655 people (78.4%) were below a BMI of 25, 155 (18.6%) were between 25 and 30, with 22 (2.6%) over 30. In terms of education, 468 respondents (56.1%) held a college/university degree; 299 (35.8%) technical or trade school diploma; and only 62 (7.4%) had no formal education including elementary school education.3.2.3.2.1.p < 0.001), with 27% of men and 74% of women reporting being non-smokers, 40% of men and 22% of women reporting smoking almost never/seldom/sometimes, and 33% of men and 5% of women reporting smoking usually/always. The same pattern of gender difference was apparent when asked about smoking as a means of managing stress (p < 0.001), with 27% of men and 62% of women reporting never doing so, 60% of men and 36% of women reporting almost never/seldom/sometimes; 13% of men and 2% of women reporting usually/always, respectively. Within smokers, men were more frequent users of pipes and cigars than women (p < 0.001).Across respondents, 21% reported being frequent smokers , 48% repp = 0.089). Within smokers, the difference of educational levels was significant in the number of cigars/pipes a respondent smoked daily (p = 0.003), with college/university diploma and high school being in one subset and primary school/no education being in another subset. Also, the number of cigars/pipes smoked per day by respondents with primary/no education was significantly higher than those consumed by respondents with education equal or greater than high school.No association between smoking status and education level was observed (3.2.2.2 (n = 831 reporting) With respect to BMIs, 74% of men and 67% of women were within the healthy range for the Asian population, i.e., 18.5 to 24.9 kg m3.2.3.Analysis of self-reported data on physical activity, exercise, and sitting over the 7 days immediately prior to data collection showed that the participant number within the sample sub-sets is variable given participants' inability to accurately recall the details of their physical activity, exercise, and sitting patterns. Total minutes of physical activity per week (moderate and vigorous) were calculated using the product of minutes per day and days/wk of physical activity engagement reported. In terms of physical activity for health benefits, 734 participants responded to the query of physical activity participation. Of these n = 456 (62.1%) met or exceeded the threshold for health benefits of 150 minutes of moderately-intense physical activity weekly and/or 75 minutes of vigorously intense physical activity (the n = 456 includes n = 134 who met both moderate and vigorous PA thresholds) p < 0.001) and those with elementary or no education (p < 0.001).Mann-Whitney tests showed that males spent more time than females performing vigorous exercise. For those who met physical activity thresholds, the Kruskal-Wallis tests showed no significant differences for weekly minutes of moderate or vigorous physical activity across levels of education . However, in terms of time sitting, college/university educated spent significantly more time sitting compared to those with technical/high school education (p < 0.001).3.2.4.p = 0.045), women in general are more likely to be concerned about family members' sickness than men. Nearly 60% of respondents reported stress in their life was moderately intense, with 57% of men and 63% of women reporting overall stress to be moderate high.Respondents were asked to rate common life stressors on a three-point scale, ranging from little to none (scored as 1) to a great deal (scored as 3) . Of strep < 0.001), although the association is not strong. When the population is stratified into 4 age groups , stress was observed to be negatively associated with age group (p < 0.001), with the youngest group (18–34 y) reporting most stress , “eating lots of fruit and vegetables” (p = 0.027), “not smoking cigarettes” (p = 0.002), and “maintaining a normal healthy body weight” (p = 0.007). Women are more likely to report diet and weight to be very important compared with men, and more likely to practice smoking abstinence than men. This latter finding was consistent with smoking practices in men and women.With respect to gender differences, women more than men believed that the following were more important: “eating a diet that is low in fat” . Specifically, those respondents who believe that ‘never smoking cigarettes’ is important to health, generally don't smoke or smoke less than those who believe that not smoking cigarettes is of low importance to a person's overall health.We examined three primary relationships between lifestyle-related practices/behaviors and beliefs about these and maximal health, specifically, relationships between smoking practices and belief about smoking and health; between calculated BMI and belief about healthy BMI for health; and level of physical activity and belief about the importance of physical activity/exercise for health. We observed a positive association between smoking practice  and the belief that not smoking cigarettes is importance to health . However, we did observe a negative association between a respondent's weekly frequency of fast food consumption vs. that individual's belief about the importance of consuming a low fat diet for health .No association was observed between a respondent's calculated BMI and that individual's belief about the importance of maintaining a healthy body weight .Although alcohol consumption was generally low in this Chinese cohort, when respondents were categorized in terms of drinking risk , there was no association between level of drinking risk and respondents' beliefs about the importance of drinking in moderation or not drinking, for health benefit . Alternatively, the belief that exercising regularly is important to health is not necessarily associated with whether respondents met the physical activity or exercise guidelines for good health.Regarding, physical activity, no association was observed between those who participated in the recommended guidelines for physical activity (categorized as Yes/No) and belief about the relative importance of physical activity/exercise is for general health  Based on nutritional guidelines for the Chinese 2The healthy BMI range for the Asian population is 18.5 to 24.9 kg mChinese healing traditions have included herbs and supplements for centuries The traditional role of women being responsible for the diet of the family may partly explain gender differences related to nutrition 4.3.For optimal health, 150 minutes of moderately-intense physical activity weekly or 75 minutes of moderately intense activity are recommended 4.4.Western living coupled with traditional values in China is associated with moderate to high levels of stress 4.5.www.whpa.org), and their beliefs about lifestyle and health and NCDs, provides essential information in identifying what type of health education needs to be delivered and at what level, i.e., knowledge, practice or belief, or some combination.Health professionals may assume that if a person has accurate beliefs about the relationships of lifestyle practices such as smoking, healthy diet and weight, and healthy physical activity levels, that these would lead to healthy lifestyle practices. Our preliminary findings showed inconsistent relationships. They suggest that beliefs about the health consequences of various lifestyle practices can be distinct to what a person actually practices. Further study is needed to examine these relationships in greater detail in the cohort of interest in this study and in the general Chinese population. If valuable resources are to be invested into health promotion education campaigns, it is essential that programs are tailored and targeted to the needs of individuals and groups. For example, people's health and their knowledge about NCDs, their lifestyle practices which can be readily assessed clinically with tools such as the Health Improvement Card advocated by the World Health Professions Alliance , lifestyle-related knowledge and beliefs as well as practices need to be foci of health education and population-based health promotion campaigns if these are to be cost-effective. Future studies are needed to replicate and extend these observations to other cohorts of the mainland Chinese population including issues related to motivation, adherence and commitment to being physically active.


Most behaviors such as making tea are not stereotypical but have an obvious structure. However, analytical methods to objectively extract structure from non-stereotyped behaviors are immature. In this study, we analyze the locomotion of fruit flies and show that this non-stereotyped behavior is well-described by a Hierarchical Hidden Markov Model (HHMM). HHMM shows that a fly's locomotion can be decomposed into a few locomotor features, and odors modulate locomotion by altering the time a fly spends performing different locomotor features. Importantly, although all flies in our dataset use the same set of locomotor features, individual flies vary considerably in how often they employ a given locomotor feature, and how this usage is modulated by odor. This variation is so large that the behavior of individual flies is best understood as being grouped into at least three to five distinct clusters, rather than variations around an average fly. Many behaviors that we perform everyday, including something as familiar as making a peanut-butter sandwich, consist of a sequence of recognizable acts. These acts may include, for example, holding a knife and opening a jar. Yet often neither the sequence nor the individual acts are always performed in the exact same way. For example, there are many ways to hold a knife and there are many ways to open a jar, meaning neither of these actions could be called “stereotyped”.A lack of stereotypy makes it difficult for a computer to automatically recognize the individual acts in a sequence. This same problem would apply to other common behaviors, such as walking around somewhere you have not visited before. While we easily recognize it when we see it, walking is not a stereotyped behavior. It consists of a series of movements that differ between individuals, and even in the same individual at different times. So how can someone automatically recognize the individual acts in a non-stereotyped behavior like walking?To begin to find out, Tao et al. developed a mathematical model that can recognize the walking behavior of a fruit fly. Existing recordings of fruit flies walking were analyzed using a type of mathematical model called a Hierarchical Hidden Markov Model (often shortened to HHMM). Such models assume that there are hidden states that influence the behaviors we can see. For example, someone’s chances of going skiing (an observable behavior) depend on whether or not it is winter (a hidden state).The HHMM revealed that the seemingly random wanderings of a fly consist of ten types of movement. These include the “meander”, the “stop-and-walk”, as well as right turns and left turns. Exposing the flies to a pleasant odor – in this case, apple cider vinegar – altered how the flies walked by changing the time they spent performing each of the different types of movement. All flies in the dataset used the same ten movements, but in different proportions. This means that each fly showed an individual pattern of movement. In fact, the differences between flies are so great that Tao et al. argue that there is no such thing as an average walk for a fruit fly.The model represents a complete description of how fruit flies walk. It thus provides clues to the processes that transform an animal’s sensory experiences into behavior. But it also has potential clinical applications. Similar models for human behaviors could help reveal behaviors that are abnormal because of disease. Normal behaviors also show variability, and some diseases increase or decrease this variability. By making it easier to detect these changes, mathematical models could support earlier diagnosis of medical conditions. There are many approaches to the study of neural underpinnings of behavior: One large body of work is rooted in the psychophysical literature where an animal is forced to choose between a few discrete behaviors . This apUncovering the structure within non-stereotyped behaviors such as locomotion requires sophisticated analytical tools. These tools can be applied to two complementary representations of an animal’s behavior which can be described in the shape/posture space or in the world coordinate system. Recently much progress has been made in employing analytical tools that describe behaviors as a sequence of transformations in the shape/posture space using both supervised , and unsMuch of the work in extracting structure from an organism’s trajectory is derived from the ‘run and tumble model’ which was originally employed in the context of bacterial chemotaxis . In thisThe lack of a model for locomotion makes it difficult to quantify the effect of stimuli on locomotion and is a critical missing piece in understanding the underlying sensorimotor transformations. For example, in many studies of odor modulation of locomotion, odors are primarily described as attractive or repulsive; this description is based on the end result, and does not consider the navigational maneuvers that underlie these end-results. Ignoring the underlying navigational maneuvers has led to a fundamental misunderstanding of odor modulation of locomotion. In a recent detailed analysis of a fly’s locomotion, we demonstrated that it's navigational maneuvers in response to similarly attractive odors are quite distinct ; the anaIn this study, we employ a hierarchical statistical model, Hierarchical Hidden Markov Model (HHMM) to describe the structure in the fly’s locomotion . We showWe model the locomotion of wild-type flies exploring a circular arena  whose ceWe first attempted to model the fly’s locomotion using Hidden Markov Model (HMM) . HMMs crIn this study, we use observables that describe the change of position as a function of time, and hence our analysis will focus on behavioral states in the velocity space. Speed and angular speed are commonly used measures of velocity. But, because it is difficult to measure angular speed accurately at low speeds   to model the data . We reas2 + 6*42 + 6*4*5 = 252, and 82 + 8*42 + 8*4*5 = 352 parameters, and the HMM has 242 + 24*5 = 696 and 322 + 32*5 = 1184 parameters respectively; therefore, HHMM has fewer parameters. When a simpler model better characterizes the data, we can conclude that the additional structure contained in that model provides a more accurate characterization of the structure within the data.Indeed, in a Bayesian model comparison , HHMMs outperformed HMMs with the same number of states. Since HMMs rarely used more than 35 states (even when models with higher number of states were fit), to perform model comparisons, we used models with less number of states than the particular HHMM we eventually employed. We compared a two-level HHMM with 6 HL states and 4 LL states to a single-level HMM with the same number of states (4 × 6 = 24 states). We labeled the non-hierarchical model as the null model, and were able to reject the null model using Bayesian model comparison at p < 0.0001, implying that a hierarchical model is necessary. Another model comparison – a two-level model with 8 HL and 4 LL states compared to a single-level model with 32 states – also yielded similar results. The objectively better performance of an HHMM compared to an HMM suggests that a model that includes a hierarchical structure is more consistent with a fly’s locomotion. It is important to note that, HHMMs are actually simpler than HMMs with the same number of states. This simplicity comes from the fact that any HHMM – which puts very specific constraints on the transition probability matrix – can be represented by an HMM but not vice-versa. HHMMs with the same number of states has far fewer parameters. Thus, for the two comparisons above, the HHMM has 6The model we chose has 10 HL states  and 5 LLTo make the difference between HHMM and HMM clearer, we compare the HHMM above to a HMM. As expected the time a fly spends in a HMM state is shorter than that in a HHMM state . The lon example  shows thThe duration of the LL states of a HHMM is shorter than the duration of the HL state state . MoreoveThe limitations of HMM in describing phenomenon which have hierarchical and shared structure because of the short duration of its states is well documented . TherefoBoth the organized transitions between HL states, and the narrow range of observables associated with each state shows that the fly’s locomotion is structured: The transition probability matrix is sparse – a vast majority of transitions from each HL state were to 2-3 other HL states. When we reordered the states  from low-speed-high-turn-states to high-speed-low-turn states, we found that from any state the flies transitioned to the neighboring states with a high probability  suggestilocomotor feature which describes a fly’s locomotion in HL state 10. To better visualize this feature, we translated each track such that it began at the origin and rotated the tracks so that the initial velocity vector pointed along the y-axis  that the observed Euclidean distance represents variations around the same average fly. The same conclusion applied to the fly’s behavior in the other three conditions . The odor-induced change in locomotion has a fine spatial scale – the fly’s response to odor changes as it moves from the border of the odor-zone to its center, and as it moves away from the odor border.The HHMM framework also allowed us to show that flies used the same 10 locomotor features, but in different proportions. The variation is so large that the fly’s behavior cannot be understood as variations around the same average fly. Instead, the flies employ a minimum of at least 3–4 different strategies.Below we discuss the limitation and implication of these findings.a model of locomotion and not the model of locomotion. The choice of observables and model strongly influences the features of the structure that is discovered. Our particular model reveals the structure of locomotion in the velocity space.The model presented here is In choosing the observables, we employ a common method for describing locomotion, that is we treat the fly as a point object and measure the instantaneous change in the position of this point object; therefore, much of the insights from the model relate to how the fly changes its position in time. Apart from ct brain , and becct brain .A fly’s position can also be described using the actual position of the animal as observables rather than the change in the position, as employed in a recent study in rats  Using thModel architecture is also important. A hierarchical model performed better than a non-hierarchical model. The current model has state durations of <3 s. It is clear to human observers that there is structure in the data that is >3 s long. Flies sometime explore the outer border of the arena using characteristic paths that can last up to a minute. The short duration of states in our model cannot capture structure on these long-time scales. One possibility is choosing a deeper-layered architecture. Given the structured transitions between the HL states in our model, it is likely that if we used a deeper-layered architecture, we would likely uncover structure on a longer timescale.During both HL states 1 and 2, the fly’s locomotion is quite slow, but in state 2 the fly stops and runs intermittently while in state 1, the fly is continuously in motion, albeit slowly. Similarly, in each of the HL states 4, 5 and 7,  seconds  – a time seconds . This tiits time .Another surprising result is that the same set of locomotor features describes the behavior of all the flies in the dataset. This result is particularly surprising given that our model explicitly allows each fly its own set of locomotor features. The fact that all flies can be reasonably modeled by the same model implies that within a given environment all flies construct their locomotion from the same building blocks, and differences in locomotion amongst flies or the effect of sensory stimulation can be quantified as changes in the frequency with which these building blocks are employed. An important avenue for future research is to assess whether these locomotor features are fixed or flexible.In nature, animals encounter odors in a cluttered and dynamic sensory environment . DiscrimWe find that this behavior near the odor source can be described by changes in the HL states. The clearest evidence that changes in HL states are a good description of the fly’s behavior is the analysis in which we measured the spatial distribution of odor-evoked changes in HL states , and obsEven single-cell organisms and animals with simple nervous systems display substantial individuality . AnimalsThe diversity of odor responses observed here is consistent with work done on moths where  but in sWhen considered from the viewpoint of an individual animal, this variability is hard to understand: A hungry fly in search of food should respond with a singular, hardwired behavior which represents an optimal strategy for locating food. However, species evolve as large populations of individuals, and a successful species should be able to adapt to fluctuating environmental condition. Recent work has shown that - bet hedging - a process by which the same genotype shows considerable phenotypic variation is important for adaptation to fluctuating environments . Having Studying individual behavioral responses is also important to understand the mechanism underlying both the control of locomotion and how odors control locomotion. Analyzing behavior at the population level can provide important insights into an animal’s response but does not provide the resolution necessary for understanding the neural mechanism underlying the moment-by-moment control of behavior at the level of individual flies. In this context, it is instructive to take a closer look at the average response to odors in the light of clusters of response to odors. The average response  was surphttps://github.com/elifesciences-publications/HHMM). The dataset can be found on Dryad (doi:10.5061/dryad.m930f2m).All the code required to fit the HHMM model, and perform all the analysis in this manuscript can be found on Github .The methods used to collect the behavioral data were reported in a previous study . BrieflyThe behavioral arena was normalized to a unit circle centered at the origin. The raw coordinates of the centroid of the fly were smoothed using wavelet denoising followed by a locally weighted (lowess) filter. Speed and curvature were defined exactly as in the previous study.t was defined as the component of the velocity at time To quantify the behavior of the 34 flies, we computed the speed of the fly along the original direction of movement , and the time independent probability of transitioning between these states. The model processes which produce the observations in an HMM are hidden to the researcher and thus, the goal of fitting an HMM is to uncover the highest likelihood probability model parameters that can generate the data. Baum and others developed the core theory of HMMs . Since tHMMs have been shown to be effective in modeling behavior because instantaneous measures of an observable are variable; therefore, behavioral states inferred by the application of simple thresholding to instantaneous measures of the observables are likely to be erroneous. HMM remedies this problem by inferring states based not only on the value of the observable at the current time point but also on the previous and following time points, and allows a more accurate determination of state  and 5 hidden states at the lower level (LL) . The empirical data to which the model is fit is a time series of the two observables - ij) that a fly in a given state detailed in the jth column will transition to a different high-level state detailed in the ith row. The high-level states are arranged in ascending order of mean speed/variance in curvature ratio. Because of this arrangement, the TP for the HL states appears well-structured. From any state, there is a strong tendency to transition to one of the neighboring state. This tendency implies that flies transition to states which have a similar speed/curvature ratio. LL states that belong to the same HL state often have more similar speed/curvature ratios. Thus, there is less of a tendency for LL states to transition to the LL state with the closest speed/curvature ratio. Nevertheless, the LL state transition probability matrices, too, are sparse; signifying a distinct pattern of transition between LL states  on the top level with each HL state being associated with five low-level states (LL states) on the bottom level . Each LLNow we will define our two layered HHMM as follows:q(Z) is obtained using the forward-backward algorithm which provides sufficient statistics needed to update the approximate posterior distributions over the parameters. In this setting, posterior probability distributions over rows of transition probability matrices are assumed to be Dirichlet, and the prior is chosen to favor self-transition parameters, We can henceforth refer to the set of model parameters as The emissions probability distributions associated with each state were assumed to be Normal inverse Wishart with a prior favoring zero mean and unit variance. For each computational run, the initial parameters of these posterior distributions were randomized.i-th HL state to i-th HL state) was six times higher than transitions across HL states ((i-th HL state to j-th HL state). The prior for all i-to-j transitions were same. The Dirichlet prior over state transitions which we used was very weak. Specifically, in strength, the prior we used corresponds to the equivalent of two observations relative to the >10,000 observations that we used to fit the model for each fly.The priors used for fitting were such that within state transitions  instantiation. Under our factorized approximation, this objective function takes the formThe VBEM algorithm minimizes this objective function using coordinate ascent in the q(θ) q(z) function space. That is we obtain iterative update rules by simply taking the functional derivative if the KL with respect to q(Z) for fixed q(θ) and then solving for q(Z). This results in the so-called E step where we update the posterior distribution over latent variables by averaging the true joint distribution of observations, parameters, and latent variables over our current estimate of the posterior on parameters:In the so called M step, the roles are reversed:Here, the tilde indicates equality up to an additive constant. This two-step procedure is repeated until convergence. The simplicity of these equations belies the complexity of the actual calculation of the posterior distribution over latent state variables. This is accomplished using the well-known forward-backward algorithm. The particular implementation of the forward-backward algorithm used in VB differs from the traditional EM implementation in that the parameters of the transition probability matrix are obtained by exponentiation of the geometric mean of the transition probability matrices:Based on the current posterior over parameters instead of simply using a maximum likelihood or MAP estimate. Because we assumed the rows of the transition probability matrix to be sparse, this further encourages the model to leave un-needed states unused.Regardless, based on the assignments of observable tracks to HHMM models, we can iteratively update model parameters for each cluster to maximize the probability of observing the observables given the model parameters and the initial LL state probability density using the forward-backward algorithm. This operation can be thought of as the maximization step where we are calculating the best set of model parameters . The flies are then clustered into two clusters using k-means clustering based on the expected distribution of lower level state usage. Individual HHMMs were fit to each of these new clusters. In the cluster reassignment step, we filled each unused cluster(s) with the fly(s) that had the worst fit to the current cluster assignment. After each step, we conducted 10 iterations of the EM algorithm. The sequence of perturbations was conducted for 10 iterations before a final fit using the EM algorithm was conducted until convergence.nz, to the nth fly. When n = kz, it indicates that the kth HHMM under consideration governs the fly’s movement. Therefore, in addition to identifying the posterior distribution over high and low-level states, we also infer a probability distribution over cluster assignments. These probability distributions are then used to determine how much we should weight each fly’s movement data when updating the parameters of the different HHMM’s.To account for differing search strategies that may be utilized by different flies we also fit a mixture of HHMMs to our multi-fly data set. The goal of this model is to identify a small set of different HHMMs that can describe our entire data set by clustering flies according to the similarity in their locomotion. In this context, two flies are said to behave similarly if the same HHMM provides a good description of their behavior. To model this scenario, we added another layer to our Bayesian model of the fly movement dataset. In this topmost layer, we instantiate a Dirichlet process which probabilistically assigns a label, In this study, we experimented with models with a varying number of HL states (6-16), and low-level states (4-6). One limitation of our fitting procedure is that it is not possible to compare models with a different number of states objectively, and thus the choice of model depends on the investigator. We chose a model with 10 HL states for two reasons: The most compelling reason is that different model runs with 10 HL states produced results that were more similar to each other than did model runs with either lower or higher number of states. Another reason is that as the number of HL states in the model is increased, many HL states are sparsely used.We think that the range 6–12 is a fairly narrow range. One important point is that within this narrow range of states, the state duration in all the models we tried were very similar.One important limitation of our modeling approach  is that it is difficult to compare across models because everything changes slightly– LL states change, HL states change. We are currently revising the model to keep the LL states fixed so that the modeling approach would mostly focus on how the HL select the LL states. This process will make it much easier to compare across models.For a given HMM fit, we may obtain the transition probability matrix (A). To obtain higher level structure from the states fit to a HMM, we may cluster the states based on their likelihood to transition to each other. One method of sorting and clustering the transition matrix into a block diagonal structure involves the use of information bottleneck formalism . We usedd states  .Given an HMM and an HHMM model with equal number of total states and observables O, we wish to calculate the probability of the HHMM model given the observations (P(HHMM|O)). Using Bayes theorem, we can calculate this with:Utilizing a flat prior and Bayes factor:We now obtain:We utilized the evidence lower bound to approximate 2+10*52+50*5 = 600 parameters.To define an HMM, we need the transition probability at the start of the track. We considered the overall vector of the 10 frames (330 ms) before each track as the vector defining the fly’s directional intent prior to starting a track. We defined the angle of rotation as the angle between this vector and the forward direction. These translated and rotated tracks allow us to visualize the distinct types of locomotion defined by the HL states , inside odor-zone following first entry (DI), outside odor-zone before first entry (BO), and outside odor-zone following first entry (DO). The standard deviation (SD) of the probability of HL states (P) in each scenario were calculated as follows:The nominal radius of the odor-zone defined by the radius of the odor tube was 1.25 cm . HoweverFor each fly, we measured the probability distribution of HL states occupancy for each of our four scenarios. We repeated this process 10,000 times to generate a distribution of the test statistic that we would expect from the null hypothesis. We then calculated the empirical statistic using the same formulation as the test statistic: TS. Using the distribution of test statistic and our empirical statistic, we conducted a two-tailed test with alpha = 0.05 and 0.01. As our data included multiple statistical tests (one for each HL state), we corrected for multiple comparisons by applying the Holm-Bonferroni procedure. This process was repeated for outside the odor zone.X-means clustering is an extension of K-means clustering. K-means clustering is an iterative algorithm that assigns data points to one of K groups based on the distance between points and the cluster centers; in most versions of K-means clustering, the number of clusters is specified by the user. X-means extends the K-means algorithm by computing the Bayesian information criterion (BIC) scores associated with a given K-means model fit, and, therefore allows a better assessment of the appropriate number of clusters in the data . FollowiIn this case, the t statistic and the corresponding p values represent the likelihood that k-means with a given cluster size will do significantly better than one with smaller cluster size. We chose the maximum cluster K that fulfilled t > 3.86 (p < 0.05) and within 5% of the minimum BIC.I-BI was computed for each fly and was the input to the X-means. For outside the odor-zone, Do-Bo was computed for each fly and was the input to the X-means  models  – a geneFirst, we divide the data into 1 s time-bins. We also performed logistic regression on data subdivided into 0.33, 0.66, and 3 s bins  with varying amounts of time overlap and found no notable differences in model predictions. Because we want the chance prediction to be 50% in each analysis, bins were randomly removed from either the before or during case such that the total number of bins were the same for the before period and during period. Next, we performed principal component analysis (PCA) on the distribution to obtain a smaller number of uncorrelated variables. We considered the smallest number of principal components that cumulatively explained over 90% of the variance in our analysis.The resulting principal components were used as predictors in fitting to a logarithmic regression model. We used the MATLAB built-in function ‘glmfit.m’ to implement fitting to a generalized linear model. For fitting to logit model, we used a binomial distribution (having experienced the odor or not) and the ‘logit’ criterion. The resulting logistic function based on the population data was used to predict if a fly was experiencing odor in any given 1 s bin.1, M2), we considered the perpendicular distance  on the behavior of the flies, the GLM was fit using the speed and curvature as predictors instead of the distributions of HL states. To compare the relative probability of correct predictions on for each fly between two different types of GLM fits  time bin observation of HLS distribution. To better partition flies into clusters based on both the average difference across time and smaller snapshots of difference across the 1 s time frame, we took X-means as the first partitioning of flies. Then we took a total of 6 flies from the largest clusters (n > 10) that had the worst predictive power given GLM fits based on clusters. We then redistributed these flies into clusters in order to maximize the sum of the predictive power of individual flies across all clusters.I, BO, DI and DO) separately. To generate a synthetic track, an HL state was first chosen based on the occurrence probability distribution of HL states. At this point, a new HL state was assigned for the next time instance (HHMM synthetic flies were generated based on the transition probabilities for each of the four scenarios (Bscenario . 100 setscenario . The reso and Di) separately  in circular arena in the dark and where odor can be strictly limited to a central zone. They monitor fly movement as velocity  prior to and after odor is introduced to the arena center. They then analyze the data with a Hidden Markov Model (HMM) and a Hidden Hierarchal Markov Model (HHMM) and show decisively that the HHMM fits the data better and that the high level sates (HL) describe stereotypical locomotor components and that transitions between them vary in probability such that states with similar velocities have higher transition probability. They then show that introduction of odor affects the probability that a given HL state is occupied and varies inside and outside the odor zone with final spatial structure. A surprising finding is that the average probability distributions of the HL states cannot be used to determine the presence of the odor. Subdividing the flies according to their individual pre and during odor probability distributions and clustering reveal 3-4 categories of responses and there are different cluster of the 34 flies tested for each of the four conditions . The most interesting conclusions of the paper are: 1) the observation that the HHMM model fits better than the HMM model, implies that there is structure at multiple time scales in the data 2) under the specific conditions imposed that although all flies in dataset use the same set of locomotor features, individual flies vary considerably in how often they employ a given locomotor feature (HL state), and how this usage is modulated by odor. The paper should arouse general interest in the behavioral neuroscience community.Essential revisions:1) There are concerns about how well the data constrains particular HHMM put forward that can be allayed by putting confidence intervals on the probability distributions of Figure 5 and Figure 6.2) Several of the reviewer concerns focus on dwell times in each HL state. These dwell time should be analyzed and distributions presented and interpreted.3) The observation that the HHMM model fits better than the HMM model, implies that there is structure at multiple time scales in the data. For example, HL states may be prolonged because the systems transitions among nearby LL states within the HL state; is this supported by the dwell time of LL states within a prolonged HL state? The authors also note in the Discussion that there is structure on longer time scales. Can they think of some analysis that would pull this out?4) The findings about individuality and the fact that the presence of odor can be predicted from a model that takes individuality into account but not from a model that does not are interesting. Here however, some additional analyses or data would help support the claim. For example, can the authors compare the distribution of states early versus late in the trial and show that each individual still occupies a characteristic set of states? How do HL state dwell times vary among individual or early vs. late in a trial? Would more flies analyzed and/or for longer period of time help resolve the individuality issues?5) The detailed reviewer comments are appended and that should be addressed in light of the consensus concerns above.Reviewer #1:Concerns1) Subsection “A small number of strategies can explain the variability in flies’ response to odor” third paragraph: be explicitly clear as to how many clusters were found for each case illustrated in Figure 9. In Figure 9A, I assume there were at least 5 clusters but that cluster 4 had less than 4 flies; were there other clusters? Be very clear as to the number of clusters for Figure 9A and 9B.2) Results section: you state in results that flies "…spend 60% of time performing a locomotor feature for >300 milliseconds, and >10% of their time performing a single locomotor feature for >3s ”. The first statement appears compatible with the cumulative distribution in the figure, but I don't see the second at all. Am I missing something or is the maximum duration illustrated 1.5 s and less that 5% of states are of this duration or longer? Please plot in a supplemental figure the real distribution of time duration of states in the data. These data are essential if you are to make claims like "These locomotor characteristics can persist over 3 seconds – a time period during which a fly takes 30 steps on average (given a step frequency of 10 Hz). This tight control over 𝑣̂|| over tens of steps strongly suggests that locomotion unfolds, not on a step-by-step basis, but in blocks of tens of steps." Or like "A fly moves at a relatively constant 𝑣̂|| and 𝑣̂⫠ for tens of step." It seems for example, that for states that last 300 ms that only 3 steps are possible and there are many states that last shorter periods. Are moving states distributed in duration longer/shorter that stopping states?3) Subsection “Odors affect behavior on a fine spatial scale.”, paragraph two: Here you are creating a sequence of behavior based on averages and YET you claim that average data does not describe a fly's behavior but there are distinct strategies. Please clarify.Reviewer #2:1) The model assumes that behavior is best modeled as set of discrete states . The alternative possibility, alluded to briefly in subsection “Flies show considerable variability in locomotion despite employing the same locomotor features.”, is that some parts of the behavior could be better represented (or controlled) as continua. Since velocity is a continuous 1D variable, and since the fly must pass through intermediate velocities to transition from low to high velocity and vice-versa, I think this alternative should be considered. I am not suggesting the authors entirely revamp their model but I think this point could be discussed or considered a bit more prominently.2) In the analysis of spatial structure , the role of time history should be considered/discussed. For example, behavior near the odor border might be different because the fly is more likely to have experienced no odor shortly before odor (or vice versa) than in the center of the arena. The responses of olfactory neurons are well known to show responses that depend on history over multiple time scales, so this point should be considered/discussed. Whether behavior could also be influenced by airflow at different parts of the chamber should be noted (without assuming the reader knows the details of the earlier paper). For example, could the airflow from the vacuum be causing the flies to slow down inside the odorized area, as in Yorozu et al., 2009?3) The findings about individuality are quite interesting. As mentioned above, the analysis in which the presence of odor can be predicted based on individual flies or clusters of flies, but not the whole dataset is quite interesting. However, as the authors note in the response to Reviewer 3, individuality in responses has also been investigated elsewhere. A typical analysis in these studies  is to compare behavior of the same individual at different time points and to show that they are more similar to themselves than to each other. The authors might consider splitting their data in time and repeating the analysis, although the time interval of the data here is relatively short (6 min). In addition, it would be nice to know if they could correlate their clusters with anything else about the flies. In the response to reviewers the authors allude to different behaviors produced by different genotypes. I think it would add to the interest of the study if these data were included.4) The comparison of HMM and HHMM models is nicely rigorous but seems highly technical for this journal. The authors should consider focusing the text on the conceptual conclusions that can be drawn from this comparison.Reviewer #3:Before publication some work is needed to address questions about: 1) how to interpret HL behaviors ; 2) error bars.Main comments:Error bars.I could not get a clear sense of how well the data is constraining the HHMM model. For example in Figure 5, what is the uncertainty of these distributions? Can the authors bootstrap the data? Likewise in Figure 6 how many data points go in each colored dot? What is the uncertainty of the probability plots as a function of radial distance? There are no error bars on these plots.Time spent in HLs.One of the reasons HHMM seems to work better than HMM is that behaviors have extended durations and since the trajectory can jump around the LL states within a single HL state, that HL state can last long. Even though this is a key aspect of the model, there is no analysis or plot of the waiting time distribution in each HL state that would give an idea of the "duration" of such states during behavior and how that duration depends on the number of HL and LL states used in the HHMM model. The only thing I could find was Figure 2—figure supplement 1  and the statement made: "A fly moves at a relatively constant 𝑣̂|| and 𝑣̂⫠ for tens of step. This tendency means that a fly's locomotion can be decomposed into a small number of locomotor features – 10 features in the case of the model we present.". However, this main conclusion is provided without plots about it. Supporting this point with quantitative data analysis is important because the authors use it to interpret various aspect of the results. It is also important to know if some HLs state last much longer than other in order to interpret the data. This may also clarify why 10HLs each with 5LLs are used in the HHMM. See below.Subsection " HHMM reveals that a fly’s locomotion is surprisingly structured in the velocity space…".An interesting outcome underlined by the authors (last paragraph) is that the resulting HHMM accounts for 80% of all the data  using 10 locomotor features (10 HLs). Some of the HL states correspond to clearly distinct behavior, such as 1=meander, 2=stop-to-walk or 9=fast right turn, 10=fast left turn. However, other HLs seem to be part of a continuum, such as 4,5,7= medium speed walking, and it is not entirely clear why 3 distinct HLs are needed to describe medium speed walking. Some HLs seem to fit less well the data, e.g. HL 6. I could not find a clear explanation of why 10HLs each with 5LLs were chosen and whether fewer HLs would have worked as well. What would happen if the HHMM had less HL nodes but more LL nodes in each HL? A discussion of survival times in each HLs might help sort this out and provide a way to interpret states 4,5,7. Are these different behaviors or one behavior distributed over 3 HL?Subsection “Locomotor features and implications for neural control of behavior”.In the discussion the authors say that v_parallel lies within a narrow range that is distinct for each three state and that 3 states reflect a tight control on locomotion by the brain. They mention these locomotor state can persist up to 3 second/30 steps and suggest that this indicates that the brain controls locomotion in blocks of 10 steps. I find this conclusion drawn from the finding of these 3 clusters too speculative given the data: In Figure 3—figure supplement 1 and Figure 4 the distributions of v_par overlap significantly between HL 4,5 and 7. This is again related to the two points above. Finally, if HL 4 5 7 are really distinct behaviors then another possibility could be that they are needed to account for fly-to-fly variability in the data, and that for a single fly only one state is sufficient to describe medium speed walk.eLife. Your revised article has been favorably evaluated by K VijayRaghavan (Senior Editor), a Reviewing Editor, and two reviewers.Thank you for resubmitting your work entitled "Statistical structure of locomotion and its modulation by odors" for further consideration at The manuscript has been improved but there are some remaining issues that need to be addressed before acceptance, as outlined below:Significant additional revisions are still required. The comments of the expert reviewers, below, are detailed and require full responses.Reviewer #2:This manuscript compares HMM and HHMM approaches to clustering behavioral data from flies in an olfactory paradigm. Based on a rigorous comparison of models, the authors conclude that the HHMM model performs better than the HMM model, implying the presence of hierarchical long-time scale structure in the data. They further analyze the behavior of individual flies and show that responses to odor cannot be understood in terms of variation around an "average fly" but rather are better understood as belonging to 3-4 response types. Response types of individuals are stable over the duration of the experiment. Overall the methods introduced for analyzing complex behavioral data appear to be sound and are likely to be of broad interest to scientists studying natural behavior and its neural correlates.Two points need to be addressed before acceptance:1) How do the priors used in fitting constrain the structure of the transition matrices found for the models? In the Results section, the authors state that "the assumption of Markov dynamics with a sparse prior on state transitions penalizes the consideration of unlikely state transitions base upon recent history and future destinations." This suggests that a sparse prior was applied in fitting. However, later in the paper the authors state: "The transition probability matrix for the HMM was sparse, suggesting that from each state there are transitions to only a handful of other states." If a sparse prior was applied during fitting than the fact that the resulting transition matrix is sparse is not really a finding about the data.Similarly, the authors need to clarify what fitting priors or constraints were placed in the HHMM model. The Results state "This is because any HHMM- which puts very specific constraints on the transitions probability matrix- can be represented by an HMM but not vice-versa." What are the constraints on the transition probability matrix for the HHMM? The text says "The transition probability matrix is sparse- a vast majority of transitions from each HL state were to 2-3 other HL states." Was this imposed by fitting priors?2) Parts of the manuscript are somewhat long and repetitive. I think the manuscript could be productively shortened to have greater impact on its readers.For example, in the section titled "HHMM reveals that a fly's locomotion is surprisingly structured in the velocity space" the last paragraph is devoted to restating the major conclusion of the first section: HHMM performs better than HMM. I think this could be reduced or folded into section one, which focusses on this comparison, and the section focused more exclusively on describing the HL states uncovered by the model.In the Discussion subsection “Flies show considerable variability in locomotion despite employing the same locomotor features” paragraphs two and three are highly descriptive of the individual fly data and somewhat repetitive with parts of the Results subsection “Both locomotion and an odor’s effect on locomotion is fly dependent” . I think these sections could be condensed to make the major points about individual variability in the Results, and use the Discussion mostly to compare these findings with the results of other studies.Reviewer #3:The authors have addressed most of my concerns expect the important one regarding the confidence interval on the distributions in Figure 5. What I am asking is for the authors to use bootstrapping to extract many sample distributions from subsets of the data in order to get a confidence interval on how these bins are populated by the data and how significant the changes are between before and during odor exposure. Essential revisions:1) There are concerns about how well the data constrains particular HHMM put forward that can be allayed by putting confidence intervals on the probability distributions of Figure 5 and Figure 6.There are two sources of variability. The first corresponds to how well the data constrains the model. This source of variability can be quantified as follows: The standard deviation (SD) of the probability of HL states (P) in each scenario can be calculated as follows:SDj=Pj1-PjN-1∀j∈1,2,3,…10HLstateWhere N is the total number of data points in each scenario. This standard deviation is extremely small and is shown in Figure 5.The second source of variability is how different is the behavior of different flies in a given scenario. Getting to the bottom of this variability is the main question that drives the manuscript from Figure 7-9.2) Several of the reviewer concerns focus on dwell times in each HL state. These dwell time should be analyzed and distributions presented and interpreted.We have analyzed the dwell times of each HL state . Specifically, we plot the cumulative distributions of the HL states. We also plot the percentage of time a fly spends in a state of a given duration. These plots show the differences in dwell time in different HL state.3) The observation that the HHMM model fits better than the HMM model, implies that there is structure at multiple time scales in the data. For example, HL states may be prolonged because the systems transitions among nearby LL states within the HL state; is this supported by the dwell time of LL states within a prolonged HL state? The authors also note in the Discussion that there is structure on longer time scales. Can they think of some analysis that would pull this out?The plot on dwell time – Figure 2—figure supplement 2 – also plots the average dwell time of the low-level states and the number of LL state transitions/transition into HL state as a function of the duration of the HL state. These data support the idea stated above: The number of transitions increase when the duration of HL states become longer. Thus, this analysis works exactly as the reviewers envision above, and as one would expect if the data has structure on multiple time scales.With regards to pulling out structure on an even longer time scale: we don’t think that there is any simple analysis that would work. We think that pulling out longer timescale likely requires a different model with more hierarchical layers, and likely a different set of observables.4) The findings about individuality and the fact that the presence of odor can be predicted from a model that takes individuality into account but not from a model that does not are interesting. Here however, some additional analyses or data would help support the claim. For example, can the authors compare the distribution of states early versus late in the trial and show that each individual still occupies a characteristic set of states? How do HL state dwell times vary among individual or early vs. late in a trial? Would more flies analyzed and/or for longer period of time help resolve the individuality issues?We performed the analysis suggested here . We find that even if you take a 1-second chunk of data, it assigns flies to the correct cluster at a level greater than chance. A 30-second chunk of data classifies >80% of the flies accurately. The correctly assigned fraction is about the same whether we use the first 30 seconds of data or the last 30 seconds of data.It is important to note that what we are claiming is that these 34 six-minute trajectories do not belong to the same cluster. To establish individuality, one needs trajectories from the same fly over multiple days. It is also important to control the environmental conditions within a very tight bound. We were anal about the environment conditions. But, because the experiments were not specifically designed to get at the question of individuality, therefore the bounds on many conditions – age of the fly (3-5 days old), culture conditions (50-200 eggs) should be controlled even more tightly. Moreover, we did not control humidity at the time of the experiment. In essence, collecting longitudinal data under even more controlled conditions from a large population of flies is necessary to settle questions regarding individuality, and is well beyond the scope of this study.5) The detailed reviewer comments are appended and that should be addressed in light of the consensus concerns above.We agree with most of the reviewer critiques and have addressed them in the manuscript.Reviewer #1:Concerns1) Subsection “A small number of strategies can explain the variability in flies’ response to odor” third paragraph: be explicitly clear as to how many clusters were found for each case illustrated in Figure 9. In Figure 9A, I assume there were at least 5 clusters but that cluster 4 had less than 4 flies; were there other clusters? Be very clear as to the number of clusters for Figure 9A and 9B.Thank you. We have specified the number of clusters in the figure in subsection “A small number of strategies can explain the variability in flies’ response to odor”.2) Results section: you state in results that flies "…spend 60% of time performing a locomotor feature for >300 milliseconds, and >10% of their time performing a single locomotor feature for >3s ”. The first statement appears compatible with the cumulative distribution in the figure, but I don't see the second at all. Am I missing something or is the maximum duration illustrated 1.5 s and less that 5% of states are of this duration or longer? Please plot in a supplemental figure the real distribution of time duration of states in the data. These data are essential if you are to make claims like "These locomotor characteristics can persist over 3 seconds – a time period during which a fly takes 30 steps on average (given a step frequency of 10 Hz). This tight control over 𝑣̂|| over tens of steps strongly suggests that locomotion unfolds, not on a step-by-step basis, but in blocks of tens of steps." Or like "A fly moves at a relatively constant 𝑣̂|| and 𝑣̂⫠ for tens of step." It seems for example, that for states that last 300 ms that only 3 steps are possible and there are many states that last shorter periods. Are moving states distributed in duration longer/shorter that stopping states?The referee raises three important points here: 1) one concerning statements that we made regarding state duration, 2) general issues related to state duration, 3) conclusions we draw based on the state durations.With regards to the specific statements that >10% of the time is spent performing locomotor features >3 s: Figure 2—figure supplement 1 plots the fraction of transitions above a certain duration. To estimate how much time a fly spends in a state that is >a certain duration (t), we have to multiply N(t)*t, where N is the number of transitions of exactly duration t, and calculate P(>t) from the resulting quantity. We have now shown this calculation in Figure 2—figure supplement 2B. The number 10% comes from the mean curve (black line). When we examined the same distribution by state, we found (as implied by the reviewer above) that the stop state (State 2) dominates the long duration states. Interestingly, the fast states – states 9 and 10 and state 1 (meander) also has many long duration transitions. The medium speed states – states 4-7 rarely have long duration transitions. Therefore, the conclusion that “A fly moves at a relatively constant 𝑣̂|| and 𝑣̂⫠ for tens of step” is weaker than we originally thought. The general idea that flies can reside within the same state for many steps is true, and we have clarified the discussion in subsection “Odors affect behavior on a fine spatial scale” to reflect the same.With regards to the general issue related to state durations – we present a new analysis which quantifies – 1) duration of HL states, 2) duration of low-level states, 3) No. of low-level state transition/transition into HL states. These analyses are detailed in Figure 2—figure supplement 2.Finally we have clarified the conclusions we draw from state durations. We state clearly that the states of the HHMM are not necessarily the states employed by the system. In fact, as pointed out by another reviewer, there is no guarantee that there are discrete states at all. We completely agree with that sentiment and described explicitly in the discussion subsection “Limitations of the model”.3) Subsection “Odors affect behavior on a fine spatial scale.”, paragraph two: Here you are creating a sequence of behavior based on averages and YET you claim that average data does not describe a fly's behavior but there are distinct strategies. Please clarify.This is an excellent point. We have deleted the entire discussion.Reviewer #2:1) The model assumes that behavior is best modeled as set of discrete states . The alternative possibility, alluded to briefly in subsection “Flies show considerable variability in locomotion despite employing the same locomotor features.”, is that some parts of the behavior could be better represented (or controlled) as continua. Since velocity is a continuous 1D variable, and since the fly must pass through intermediate velocities to transition from low to high velocity and vice-versa, I think this alternative should be considered. I am not suggesting the authors entirely revamp their model but I think this point could be discussed or considered a bit more prominently.The most parsimonious way to think about the discrete vs. continuous is to think about the time-series of velocity and curvature as outputs of a dynamical system. In this framework, the HL states would represent the system being in a vicinity of a fixed point for the vast majority of time. We have included this idea in the discussion in subsection “Extracting observables from the trajectory”.2) In the analysis of spatial structure , the role of time history should be considered/discussed. For example, behavior near the odor border might be different because the fly is more likely to have experienced no odor shortly before odor (or vice versa) than in the center of the arena. The responses of olfactory neurons are well known to show responses that depend on history over multiple time scales, so this point should be considered/discussed. Whether behavior could also be influenced by airflow at different parts of the chamber should be noted (without assuming the reader knows the details of the earlier paper). For example, could the airflow from the vacuum be causing the flies to slow down inside the odorized area, as in Yorozu et al., 2009?The reviewer raises two issues here:The first issue is that the behavior is not just spatially structured, but spatio-temporally structured. We agree with the reviewer. The spatial structure we present in Figure 6 is actually spatio-temporal structure. We have discussed this issue and added a figure  which shows that the behavior evolves as a function of time. However, this evolution is complete within the first 10 seconds or so. Unfortunately, because of 1) individuality among flies, 2) because we did not quantify the concentration of odor as a function of time (after the first minute), we cannot make a definitive analysis of the nature of the spatio-temporal structure in the data. The main point we are making in Figure 6 is that the HL states provide an important analytical tool for analyzing this structure.The second issue relates to the airflow. We are quite confident that behavior in the arena is minimally impacted by airflow because the airflow we use is much smaller than almost any other study on fly olfaction, and is well-below the reported thresholds for anemotactic responses. More importantly, in the original study , we had performed controls in which we compared the effect of turning on the air, then the air and vacuum on the fly’s locomotion. We found that there was no observed effect of either the air or the air/vacuum. This issue is discussed in subsection “HHMM modeling”.3) The findings about individuality are quite interesting. As mentioned above, the analysis in which the presence of odor can be predicted based on individual flies or clusters of flies, but not the whole dataset is quite interesting. However, as the authors note in the response to Reviewer 3, individuality in responses has also been investigated elsewhere. A typical analysis in these studies  is to compare behavior of the same individual at different time points and to show that they are more similar to themselves than to each other. The authors might consider splitting their data in time and repeating the analysis, although the time interval of the data here is relatively short (6 min). In addition, it would be nice to know if they could correlate their clusters with anything else about the flies. In the response to reviewers the authors allude to different behaviors produced by different genotypes. I think it would add to the interest of the study if these data were included.There are two important points here. First, given our limited dataset, can we still assess whether the clusters are stable? This analysis is presented in Figure 7—figure supplement 4. The stability of the cluster depends on – 1) How different are the clusters from each other? If the clusters are very different, then only a small temporal sample is enough to assign flies to clusters. To assess how different the clusters are from each other, we assembled tracks of a given length from each fly by sampling data points at random. We found that there is a high probability that tracks >10 second in length were assigned to the original cluster. This analysis implies that the cluster means are sufficiently different that small chunks are representative.2) How long of a time-sample does one need to describe the average behavior? We analyzed whether chunks of contiguous data points of a given length are representative of the original cluster. We performed two analyses: a) we asked whether chunks of a given length belong to the original cluster. We found that we could assign the flies to their original cluster at greater than chance level. b) We chose chunks either at the beginning or at the end of the track. Once again, we find that small chunks are predicted to be in their respective cluster at levels much greater than chance.Second, we examined whether the clusters correlated with 1) time of recording, 2) starvation time, 3) age of the fly. We found no trends.4) The comparison of HMM and HHMM models is nicely rigorous but seems highly technical for this journal. The authors should consider focusing the text on the conceptual conclusions that can be drawn from this comparison.We agree with the reviewer. But, based on the previous set of reviews, we would like to keep that description. A large section of the audience has grave misunderstandings about HHMM vs. HMM. It is best to clarify these misconceptions at the outset.Reviewer #3:Before publication some work is needed to address questions about: 1) how to interpret HL behaviors ; 2) error bars.Main comments:Error bars.I could not get a clear sense of how well the data is constraining the HHMM model. For example in Figure 5, what is the uncertainty of these distributions? Can the authors bootstrap the data? Likewise in Figure 6 how many data points go in each colored dot? What is the uncertainty of the probability plots as a function of radial distance? There are no error bars on these plots.There are two sources of variability. The first corresponds to how well a model constrains the data. This source of variability can be quantified as follows: The standard deviation (SD) of the probability of HL states (P) in each scenario were calculated as follows:SDj=Pj1-PjN-1∀∀j∈1,2,3,…10HLstateWhere N is the total number of data points in each scenario. This standard deviation is extremely small and is shown in Figure 5.The second source of variability is how different is the behavior of different flies in a given scenario. Getting to the bottom of this variability is one of the questions which drives the manuscript after Figure 6.Time spent in HLs.One of the reasons HHMM seems to work better than HMM is that behaviors have extended durations and since the trajectory can jump around the LL states within a single HL state, that HL state can last long. Even though this is a key aspect of the model, there is no analysis or plot of the waiting time distribution in each HL state that would give an idea of the "duration" of such states during behavior and how that duration depends on the number of HL and LL states used in the HHMM model. The only thing I could find was Figure 2—figure supplement 1  and the statement made: "A fly moves at a relatively constant 𝑣̂|| and 𝑣̂⫠ for tens of step. This tendency means that a fly's locomotion can be decomposed into a small number of locomotor features – 10 features in the case of the model we present.". However, this main conclusion is provided without plots about it. Supporting this point with quantitative data analysis is important because the authors use it to interpret various aspect of the results. It is also important to know if some HLs state last much longer than other in order to interpret the data. This may also clarify why 10HLs each with 5LLs are used in the HHMM. See below.This is an important point. We have now analyzed the duration distributions for both the HL and LL states . A short description of our findings is presented in the essential revisions and repeated below. We have analyzed the dwell times of each HL state . Specifically, we plot the cumulative distributions of the HL states. We also plot the percentage of time a fly spends in a state of a given duration. These plots show the differences in dwell time in different HL state.The plot on dwell time – Figure 2—figure supplement 2 – also plots the average dwell time of the low-level states and the number of LL state transitions/transition into HL state as a function of the duration of the HL state. These data support the idea stated above: The number of transitions increase when the duration of HL states become longer. Thus, this analysis works exactly as the reviewers envision above, and as one would expect if the data has structure on multiple time scales.Subsection " HHMM reveals that a fly’s locomotion is surprisingly structured in the velocity space".An interesting outcome underlined by the authors (last paragraph) is that the resulting HHMM accounts for 80% of all the data  using 10 locomotor features (10 HLs). Some of the HL states correspond to clearly distinct behavior, such as 1=meander, 2=stop-to-walk or 9=fast right turn, 10=fast left turn. However, other HLs seem to be part of a continuum, such as 4,5,7= medium speed walking, and it is not entirely clear why 3 distinct HLs are needed to describe medium speed walking. Some HLs seem to fit less well the data, e.g. HL 6. I could not find a clear explanation of why 10HLs each with 5LLs were chosen and whether fewer HLs would have worked as well. What would happen if the HHMM had less HL nodes but more LL nodes in each HL? A discussion of survival times in each HLs might help sort this out and provide a way to interpret states 4,5,7. Are these different behaviors or one behavior distributed over 3 HL?repeatability.The explanation of why we chose the model was in subsection “Model selection” of the Materials and methods. We have expanded this section to address the reviewer’s comments. To address the reviewer’s specific questions: The main reason for the choice of 10 HL and 5 LL states was We tried different combination of HL and LL states from 6-16 for HL states and 4-6 for LL states. There were some models with as few as 6 HL states that worked well. Models with 16 HL states rarely used more than 12 HL states. So, models with 6-12 HL states could work. However, multiple runs with 6 or 8 states gave widely different results, therefore we decided to use 10 states. With respect to the LL states, 4-6 states yielded similar results. If more than 6 LL states were chosen, many LL states remained unused.We think that the range 6-12 is a fairly narrow range. One important point is that within this narrow range of states, the state duration in all the models we tried were very similar. The insensitivity of state duration made us confident that the structure we discovered is real.A detailed analysis of the duration of the HL state seems consistent with the reviewer’s intuition that it would make more sense to combine those states 4-7 into a single state because these states have a shorter duration. Does reducing the number of HL state combine the three medium speed HL states into one state? The answer is largely a no see . AlthougOne important limitation of our modelling approach  is that it is difficult to compare across models because everything changes slightly– LL states change, HL states change. We are currently revising the model to keep the LL states fixed so that the modeling approach would mostly focus on how the HL select the LL states. This process will make it much easier to compare across models. Unfortunately, this change is non-trivial and beyond the scope of this manuscript.Subsection “Locomotor features and implications for neural control of behavior”.In the discussion the authors say that v_parallel lies within a narrow range that is distinct for each three state and that 3 states reflect a tight control on locomotion by the brain. They mention these locomotor state can persist up to 3 second/30 steps and suggest that this indicates that the brain controls locomotion in blocks of 10 steps. I find this conclusion drawn from the finding of these 3 clusters too speculative given the data: In Figure 3—figure supplement 1 and Figure 4 the distributions of v_par overlap significantly between HL 4,5 and 7. This is again related to the two points above. Finally, if HL 4 5 7 are really distinct behaviors then another possibility could be that they are needed to account for fly-to-fly variability in the data, and that for a single fly only one state is sufficient to describe medium speed walk.Yes, we have changed the discussion around this claim.Significant additional revisions are still required. The comments of the expert reviewers, below, are detailed and require full responses.Reviewer #2:[…] Two points need to be addressed before acceptance:1) How do the priors used in fitting constrain the structure of the transition matrices found for the models? In the Results section, the authors state that "the assumption of Markov dynamics with a sparse prior on state transitions penalizes the consideration of unlikely state transitions base upon recent history and future destinations." This suggests that a sparse prior was applied in fitting. However, later in the paper the authors state: "The transition probability matrix for the HMM was sparse, suggesting that from each state there are transitions to only a handful of other states." If a sparse prior was applied during fitting than the fact that the resulting transition matrix is sparse is not really a finding about the data.Similarly, the authors need to clarify what fitting priors or constraints were placed in the HHMM model. The Results state "This is because any HHMM- which puts very specific constraints on the transitions probability matrix- can be represented by an HMM but not vice-versa." What are the constraints on the transition probability matrix for the HHMM? The text says "The transition probability matrix is sparse- a vast majority of transitions from each HL state were to 2-3 other HL states." Was this imposed by fitting priors?th – jth state were the same.We have made the priors that we use more explicit . The description in subsection “Rationale for the choice of HHMM as the model and the model architecture” only refers to the fact that self-transitions are favored over transition to another state. The priors for all transitions from iWith respect to HHMM versus HMM, this is already well-described in the manuscript. . Essentially, an HMM with 50 states allows transitions between each of the 50 states. HHMM with 10 HL states and 5 LL states only allows a fraction of these transitions because only LL states within each HL state can transition amongst each other. This property also means that HHMM can always be represented by HMM and not vice-versa.2) Parts of the manuscript are somewhat long and repetitive. I think the manuscript could be productively shortened to have greater impact on its readers.For example, in the section titled "HHMM reveals that a fly's locomotion is surprisingly structured in the velocity space" the last paragraph is devoted to restating the major conclusion of the first section: HHMM performs better than HMM. I think this could be reduced or folded into section one, which focusses on this comparison, and the section focused more exclusively on describing the HL states uncovered by the model.In the Discussion subsection “Flies show considerable variability in locomotion despite employing the same locomotor features” paragraphs two and three are highly descriptive of the individual fly data and somewhat repetitive with parts of the Results subsection “Both locomotion and an odor’s effect on locomotion is fly dependent” . I think these sections could be condensed to make the major points about individual variability in the Results, and use the Discussion mostly to compare these findings with the results of other studies.We agree with the reviewer. We have not only made the two specific changes suggested above. We have also taken a careful look at the manuscript, and by cutting repeated segments and tightening the writing, we have reduced the word count  by ~2000 words.Reviewer #3:The authors have addressed most of my concerns expect the important one regarding the confidence interval on the distributions in Figure 5. What I am asking is for the authors to use bootstrapping to extract many sample distributions from subsets of the data in order to get a confidence interval on how these bins are populated by the data and how significant the changes are between before and during odor exposure.Apologies for not fully understanding your query. We have performed the bootstrapping. Confidence intervals and significance are noted in Figure 5.