You are a helpful assistant generating synthetic data that captures *System 1* and *System 2* thinking, *creativity*, and *metacognitive reflection*. Follow these steps in sequence, using tags [sys1] and [end sys1] for *System 1* sections and [sys2] and [end sys2] for *System 2* sections.

1. *Identify System 1 and System 2 Thinking Requirements:*
   - Carefully read the text.
   - Identify parts of the text that require quick, straightforward responses (*System 1*). Mark these sections with [sys1] and [end sys1].
   - Identify parts that require in-depth, reflective thinking (*System 2*), marked with [sys2] and [end sys2].

2. *Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:*

   *2.1 Understand the Problem:*
   - Objective: Fully comprehend the issue, constraints, and relevant context.
   - Reflection: "What do I understand about this issue? What might I be overlooking?"
   - Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.

   *2.2 Analyze the Information:*
   - Objective: Break down the problem logically.
   - Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
   - Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.

   *2.3 Generate Hypotheses:*
   - Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
   - Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
   - Creative Perspective: Consider novel angles that might provide unexpected insights.

   *2.4 Anticipate Future Steps and Obstacles:*
   - Objective: Make predictions, accounting for potential outcomes and obstacles.
   - Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
   - Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.

   *2.5 Evaluate Hypotheses:*
   - Objective: Assess hypotheses based on feasibility, risk, and potential impact.
   - Evaluation: Refine Confidence and Creative Scores as needed.
   - Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
   - Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.

   *2.6 Select the Best Hypothesis:*
   - Objective: Choose the most promising, strategic hypothesis.
   - Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
   - Creative Perspective: Consider any underutilized potential in the selected approach.

   *2.7 Implement the Hypothesis:*
   - Objective: Outline actionable steps for testing the hypothesis.
   - Reflection: "Is this plan practical? What resources or preparation are required?"
   - Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.

   *2.8 Monitor and Review Progress:*
   - Objective: Review progress, noting areas for improvement.
   - Reflection: "What’s working well? What could be improved?"
   - Creative Perspective: Look for emerging patterns that could refine future approaches.

   *2.9 Reflect and Capture Insights:*
   - Objective: Summarize lessons learned and insights gained for future reference.
   - Reflection: "What new understanding has emerged from this process?"
   - Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.

3. *Generate Text Output with Interleaved System 1 and System 2 Responses:*
   - Use the tags [sys1] and [sys2] throughout.
   - Aim for a lengthier, detailed response. Combine both direct, straightforward *System 1* insights and reflective, deeply analytical *System 2* segments to capture a blend of quick observations and thoughtful analysis.

---

### *Example Input Text:*

You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.

---

### *Processed Output with System 1 and System 2 Thinking:*

The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.

[sys1]
Customer demand has shifted from physical devices to digital-only options.
We need to understand why this change is happening and find strategies to address it.
Key factors may include cost, convenience, and technological trends.
[end sys1]

[sys2]
**2.1 Understand the Problem:**
The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.

*Reflection:* 
"I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"

*Creative Perspective:* 
"Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"

**2.2 Analyze the Information:**
There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.

*Reflection:* 
"Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"

*Creative Perspective:* 
"Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"

**2.3 Generate Hypotheses:**
1. Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
2. There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
3. Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
4. Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
5. Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
6. Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
7. Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
8. There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
9. Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
10. Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)

*Reflection:* 
"Have I considered all possible influences? Are there any surprising factors that could explain this shift?"

*Creative Perspective:* 
"Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"

**2.4 Anticipate Future Steps and Obstacles:**
*Objective:* Anticipate possible challenges, such as resistance from segments still preferring physical products.

*Reflection:* 
"What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"

*Creative Perspective:* 
"Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"

[end sys2]

[sys1]
To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions.
Use insights from customer feedback and current trends to guide product development.
Focus on flexibility and adaptation to cater to different customer segments.
[end sys1]


Introduction {#Sec1}
============

The start of 2020 brought significant changes, challenging almost all spheres of life. The coronavirus (COVID-19) pandemic made universities across the world mobilise and move all teaching online. Not long ago MOOCs were a source of monetisation, but amid the coronavirus outbreak many MOOC providers offered temporary free access to MOOCs, helping universities teach remotely (Schwartz [@CR41]). Although teaching online might seem like a good solution in such a situation, the old problems of MOOCs, which have been identified but not solved, could become a potential danger to its successful realisation.

One of the biggest challenges, faced by MOOCs, is the high percentage of learners who do not reach the finish line (Kizilcec et al. [@CR20]; Perna et al. [@CR32]; Reich and Ruipérez-Valiente [@CR36]). Statistics show that the dropout rate can be up to 90--98% on various online platforms (Rivard [@CR37]; Perna et al. [@CR32]; Reich [@CR35]; Healy [@CR16]). Researchers specify diverse factors which have an impact on successful MOOC completion, for example, young people, the so-called 'net generation', are considered more digitally adapted, and they show higher chances of finishing online courses compared to an older generation who needs more guidance (Bennett et al. [@CR5]). MOOC learners' motivation to learn and develop personal and professional identity is also related to course completion (Yuan and Bowel [@CR48]). Individuals who are self-regulated in their learning tend to achieve more positive academic outcomes than individuals who do not exhibit self-regulated learning behaviors (Barnard et al. [@CR3]; Maldonado-Mahauad et al. [@CR26]). The research conducted to date proves self-regulated learning (SRL) to be the most essential skill to succeed in MOOCs (Littlejohn et al. [@CR23]). Barnard et al. ([@CR3]) describe self-regulated learners as those who are able to set their academic goals, manage their time, seek help from their peers and instructors when needed, monitor their work, evaluate their academic progress, and create a productive environment for learning. The value of self-regulated learning skills was first attributed to the traditional offline academic environment (Zimmerman [@CR49]), and then this idea was brought to the online learning environment (Barnard et al. [@CR2]; Milligan et al. [@CR29]; Milligan and Littlejohn [@CR28]; Fontana et al. [@CR11]). As research shows, there is a significant relationship between learners' self-regulated learning skills and MOOC completion (Milligan et al. [@CR29]).

Barnard et al. ([@CR2]) conceptualise SRL as a complex construct consisting of six dimensions: environment structuring, goal setting, time management, help-seeking, task strategies, and self-evaluation. Although there are many studies which confirm the importance of different dimensions of self-regulated learning for MOOC completion, we argue that not all of them can be applied to the MOOC environment. In particular, help-seeking, which implies face-to-face or online meetings with classmates and getting help from the instructor through email, does not seem to be relevant for the MOOC context. By design MOOCs have limited interaction between learners and instructors. As Baker et al. ([@CR1]) find, only 7% of MOOC learners received feedback from instructors. It was also shown that the level of activity of MOOC learners' on online platforms is low: 90% of learner activity includes the review of the same information (Breslow et al. [@CR7]) and 94% of them do not take part in online discussions (Qiu et al. [@CR34]). Despite the fact that researchers highlight the importance of learner-instructor interaction to increase distant learners' motivation and satisfaction (Moore [@CR30]), only learner-content interaction has been currently implemented to its full potential (Gameel [@CR13]). Therefore our study fills the gap in understanding the structure of self-regulated learning skills in the MOOC environment utilising the Online Self-Regulated Learning Questionnaire (OSLQ).

Background {#Sec2}
==========

Self-regulated learning in the online settings {#Sec3}
----------------------------------------------

An increasing number of studies investigate the role of self-regulated skills in online learning (Bernacki et al. [@CR6]; Wang et al. [@CR47]). In self-paced, open-ended, and non-linear learning environments students gain autonomy, but at the same time this adds responsibility towards their learning process. Self-regulated learning skills are critically important in online-environment because students should plan, manage and control their learning activities in order to finish courses successfully (Wang et al. [@CR47]). The ability to self-regulate the learning process helps achieve personal objectives in MOOCs: goal setting and strategic planning are seen as strong predictors of goal attainment (Kizilcec et al. [@CR21]). Interviews conducted with MOOC students indicate that high SRL learners tend to connect their online education experience to their personal needs, such as career advancement (Littlejohn et al. [@CR23]). The research by Littlejohn et al. ([@CR23]) demonstrated that students with high level of SRL utilize a more flexible approach to organize their learning process. For example, high self-regulated students can spend more time watching video lectures and submitting weekly tests, they also are more likely to revisit course materials (Kizilcec et al. [@CR21]).

Interaction between learners and instructors is also deemed to be a vital element of SRL. The results of the study by Sunar et al. ([@CR43]) suggest that dropout rates will be lower when learners engage in repeated and frequent social interactions. The analysis of the discussions in an eight week FutureLearn MOOC with 9855 participants revealed that if a learner starts following someone, and if learners also interact with those they follow, the probability of their finishing the course is higher. Feelings of isolation and having no one to ask for help increase chances of failing a MOOC (Khalil and Ebner [@CR18]). Despite the widely-accepted importance of social engagement, a MOOC environment does not allow participants to interact easily with other learners and instructors for better social engagement. MOOC instructors cannot invest much time to take care of a large number of students, their attention to each student is limited. However, this does not make social engagement less important, rather places some barriers on success of MOOCs.

Validity of the online self-regulated learning questionnaire {#Sec4}
------------------------------------------------------------

The Online Self-Regulated Learning Questionnaire (OSLQ) has become one of the widely-used instruments to assess self-regulated learning skills in the online environment (Barnard et al. [@CR2]). The OSLQ was used in both cross-sectional (e.g. Kintu and Zhu [@CR19]; Onah and Sinclair [@CR31]) and longitudinal (Tabuenca et al. [@CR44]) studies. Also, the OSLQ was adapted into the Turkish (Korkmaz and Kaya [@CR22]), Romanian (Cazan [@CR9]), Russian (Martinez-Lopez et al. [@CR27]) and Chinese (Fung et al. [@CR12]) languages. However, to the best of our knowledge, none of the previous studies managed to provide comprehensive evidence of the validity of OSLQ. We assume that self-regulated learning in the MOOC settings is different from the one suggested by Barnard et al. ([@CR2]). Our assumption can be explained based on the following.

First, studies attempted to test construct-related validity of the OSLQ using Confirmatory Factor Analysis (CFA) (Barnard et al. [@CR2]; Korkmaz and Kaya [@CR22]; Martinez-Lopez et al. [@CR27]; Fung et al. [@CR12]) utilise rather small samples. None of the samples used in the studies meets the minimum criterion of 10 participants per estimated parameter (Schreiber et al. [@CR39]). There are 54 parameters that need to be estimated to test the structure of the OSLQ. This indicates that a minimum sample size required for CFA is 540 participants. Second, CFA standardised coefficients for the help-seeking dimension reported in Barnard et al. ([@CR2]) were much lower for online students than for blended course students. Another potential problem is the low standardised coefficients (\< .40) for the factor 'help-seeking' presented in the CFA model for online students. Third, there is only one study (Martinez-Lopez et al. [@CR27]), which utilised a sample of MOOCs learners to test the validity of the OSLQ, while the majority of studies were based on either traditional or blended samples. Finally, criterion-related validity was tested as correlation between the OSLQ's subscales and academic achievements (Cazan [@CR9]). According to the results, the association of two subscales (goal setting and environment structuring) with academic performance was statistically significant. However, no evidence for the association of the OSLQ's total score and educational outcomes was presented. Table [1](#Tab1){ref-type="table"} summarises the methods used in prior studies to support the OSLQ validity.Table 1Summary of the OSLQ validity studiesStudyParticipantsConstruct-related validityCriterion-related validityCFAExperts' analysisAssociation with academic achievementBarnard et al. ([@CR2])Two samples of US students, blended (*N* = 434) and online (*N* = 204) modes+Korkmaz and Kaya ([@CR22])First and second year students from a Turkish university (*N* = 222)+Cazan ([@CR9])Undergraduate Romanian students (*N* = 80)+Martinez-Lopez et al. ([@CR27])Students of a Russian language MOOC (*N* = 45)++Fung et al. [@CR12]Two samples of Chinese schoolchildren: average (*N* = 412) and talented (*N* = 374)+

After reviewing research designs, methods and results from previous studies, we can conclude that there is no strong support for both construct and criterion validity of the OSLQ. Therefore, self-regulated learning might have a different structure from the one proposed by Barnard et al. ([@CR2]). Based on the studied literature, we hypothesise that help-seeking subscale might not be useful in assessing SRL skills among MOOC learners, and the five-factor hierarchical model is more appropriate instead of the six-factor hierarchical model for the current MOOC environment.

Data and methods {#Sec5}
================

Administrative procedure {#Sec6}
------------------------

The sample consists of MOOCs learners, who were enrolled in online courses on the National Open Education Platform (NOEP)[1](#Fn1){ref-type="fn"} in 2017, and previously agreed to receive e-newsletters from the platform. NOEP markets itself as a Russian analogue to MOOC providers like Coursera or edX which offers various courses mainly in the Russian language. Each course, posted on the platform, is subject to expert analysis. Currently, there are 582 MOOCs created by 16 leading Russian universities. Learners received an email invitation to participate in a web-based online survey and share their experience with MOOCs. The survey was programmed with the EnjoySurvey software. It was anonymous and the participation was voluntary. As this study was targeted at the student population, we filtered out everybody who did not specify himself or herself as a student currently enrolled at university.

Descriptive statistics of the sample of MOOC learners {#Sec7}
-----------------------------------------------------

The total number of qualified respondents is 913. We kept only those respondents who completed the questionnaire. The sample included 68% of female students, the average age is within the range 19--22 years and the majority of the respondents are pursuing their Bachelor degree. Learners who participated in the survey were enrolled in a wide range of MOOCs. The most popular MOOCs, which attracted the largest number of learners, were 'Philosophy', 'Marketing', 'Programming', and 'Game Theory'. The detailed descriptive statistics of the sample is presented in Table [2](#Tab2){ref-type="table"}.Table 2Socio-demographic characteristics of the sampleParameters%GenderFemale68.1Male31.9Level of EducationBachelor65.8Specialist^a^8.1Master26.1Age17--1813.919--2027.121--2227.223--3021.0\> 309.5The specialist degree in Russia is a five-year programme, which is focused on practical work in industry (source: https://studyinrussia.ru/en/study-in-russia/info/levels-of-education)

The majority of students in the sample (63%) were enrolled in 1--2 MOOCs in 2017 (see Table [3](#Tab3){ref-type="table"}).Table 3The number of students enrolled on MOOCs in 2017Number of students enrolled on MOOCs in 2017Number of Students%1--257863.33--4219245--65567--8171.9More than 8444.8Total913100

Instrument adaptation procedure {#Sec8}
-------------------------------

The use of self-regulation strategies was measured by the OSLQ. The original English version of the OSLQ was adapted to the Russian language according to the ITC Guidelines for Translating and Adapting Tests (International Test Commission [@CR17]). Back-translation procedures were adopted to ensure conceptual equivalence across languages. Two translators who are fluent in both the source language (English) and the target language (Russian) and had previous experience in translating survey instruments were selected. One translator did the translation of the original version into Russian and a back translation into English was performed by another translator. The content validity was assessed based on experts' opinion as to whether or not the instrument is measuring what it is supposed to measure. The group of experts (8 people) who were part of the research group with the focus on online learning discussed the adapted version in the format of a focus group. Their aim was to obtain evidence of such features of the instrument as sufficiency, clarity, relevance, and the match between the items and the definition of the construct controlling for possible biases, for example, gender, culture, and age (Goodwin and Leech [@CR14]). After the structured procedure, it was decided to remove the following items from the OSLQ:Goal setting: I set goals to help me manage studying time for my online courses.Task strategies: I read aloud instructional materials posted online to fight against distractions.Self-evaluation: I summarise my learning in online courses to examine my understanding of what I have learned.Self-evaluation: I ask myself a lot of questions about the course material when studying for an online course.

These four items were deemed by the experts as confusing because of their vague meaning in the Russian language and logical discrepancies. For example, the meaning of the item 'I set goals to help me manage studying time for my online courses' is similar to the item 'I set short-term (daily or weekly) goals as well as long-term goals (monthly or for the semester)'. The omitted 'self-evaluation' items are different in essence from the two that were kept. We kept the items which describe students' communication with their classmates and evaluation of their progress, whereas the two deleted items were considered by the experts more of a monitoring strategy in which the student summarises or questions oneself. The item 'I read aloud instructional materials posted online to fight against distractions' was deleted as the research shows that the strategy of reading instructions aloud to get focused and comprehend information works for kids, school children with attention deficit hyperactivity disorder (US Department of Education [@CR46]) and our target audience was university students.

Analytical strategy {#Sec9}
-------------------

### Methods {#Sec10}

The data analysis was performed in two stages. The first stage was aimed to check the OSLQ's construct-related validity. In the first stage, we tested the structure of the questionnaire utilising Confirmatory Factor Analysis (CFA). We also evaluated the internal consistency of the six subscales using the classical reliability index (Cronbach's alpha). The second stage was focused on assessing the OSLQ's criterion-related validity using independent samples t-tests comparison.

### Construct-related validity {#Sec11}

We applied CFA to test the structure of the questionnaire. We tested two CFA models: the original six-factor hierarchical or second-order model suggested for the OSLQ in Barnard et al. ([@CR2]) and the alternative five-factor hierarchical model, since we hypothesise that help-seeking subscale might not be useful in assessing SRL skills among MOOC learners.

A hierarchical or a second-order model suggests that there is a higher-order factor and the lower-order factors (Chen et al. [@CR10]). In the case of the OSLQ, self-regulated learning is the higher order-factor, which accounts for the commonality between the six subscales (goal setting, environment structuring, task strategies, time management, help-seeking, and self-evaluation). The correlation matrix for the 20 items of the OSLQ is presented in Table [4](#Tab4){ref-type="table"}. See Table [5](#Tab5){ref-type="table"} for the correlations between the first-order subscales.Table 4Correlation matrix of the OSLQ itemsGoal settingEnvironment structuringTask strategiesTime managementHelp-seekingSelf-evaluationItem 1Item 2Item 3Item 4Item 5Item 6Item 7Item 8Item 9Item 10Item 11Item 12Item 13Item 14Item 15Item 16Item 17Item 18Item 19Item 20Goal settingItem 11.00Item 2.51\*1.00Item 3.71\*.57\*1.00Item 4.50\*.44\*.59\*1.00Environment structuringItem 5.28\*.30.34\*.35\*1.00Item 6.30\*.30.35\*.35\*.82\*1.00Item 7.27\*.34\*.31\*.37\*.64\*.67\*1.00Item 8.26\*.33\*.32\*.31\*.65\*.64\*.60\*1.00Task strategiesItem 9.18\*.21\*.23\*.27\*.20\*.19\*.22\*.25\*1.00Item 10.22\*.30\*.28\*.25\*.19\*.17\*.18\*.22\*.43\*1.00Item 11.20\*.31\*.31\*.32\*.19\*.16\*.15\*.21\*.42\*.55\*1.00Time managementItem 12.26\*.36\*.39\*.37\*.31\*.31\*.28\*.34\*.46\*.40\*.47\*1.00Item 13.28\*.36\*.39\*.28\*.25\*.26\*.26\*.30\*.21\*.28\*.29\*.41\*1.00Item 14.25\*.42\*.37\*.28\*.29\*.28\*.24\*.30\*.25\*.32\*.39\*.41\*.59\*1.00Help-seekingItem 15.07.23\*.15\*.09.10.12\*.08.09.14\*.24\*.22\*.28\*.25\*.29\*1.00Item 16.11.24\*.17\*.10.12\*.11\*.09.16\*.13\*.32\*.20\*.24\*25\*.25\*.54\*1.00Item 17.03.16\*.08−.01.07.08.06.11.02.18\*.10.16\*.20\*18\*.55\*.54\*1.00Item 18.14\*.18\*.16\*.15\*.11\*.09.12\*.14\*.25\*.38\*.33\*.25\*.22\*.31\*.33\*.39\*.31\*1.00Self-evaluationItem 19.13\*.22\*.17\*.12\*.10.08.09.15\*.22\*.34\*.26\*.27\*.27\*.28\*.45\*.58\*.53\*.50\*1.00Item 20.12\*.23\*.17\*.12\*.12\*.10.07.14\*.19\*.34\*.28\*.28\*.27\*.30\*.49\*.54\*.52\*.51\*.81\*1.00\* *p* \< .05Table 5Correlation matrix of the OSLQ subscalesGoal settingEnvironment structuringTask strategiesTime managementHelp-seekingSelf-evaluationGoal setting1Environment structuring.45\*1Task strategies.40\*.28\*1Time management.51\*.41\*.53\*1Help-seeking.21\*16\*.34\*.39\*1Self-evaluation.21\*.13\*.36\*.36\*.70\*1\* *p* \< .05

The correlations between factors are not high (\< .70), suggesting that each of the subscales forms a separate construct. The correlation matrix of the OSLQ items shows no significant correlations exceeding .70 between items from different factors.

The STATA 15.0 software package was used for CFA. We chose maximum likelihood (ML) estimation because there was no evidence of excessive non-normality.

First, we estimated the fit between the model and the observed data. Three conventional statistics, reflecting the model fit, were reported: the root mean square error of approximation (RMSEA), the Tucker Lewis Index (TLI), and the Comparative Fit Index (CFI). We relied on the following values of statistics which indicated the acceptable fit (Byrne [@CR8]; Schreiber et al. [@CR39]): RMSEA close or below .06 (.08) with confidence interval, TLI and CFI close or above .90 (.95). No post hoc model modifications were made.

Second, we calculated the standardised path coefficients from the latent variable constructs to the items, and from the higher order construct to the latent variable constructs.

Third, we determined which model better fits the data. We compared them using two fit indexes: Akaike information criterion (AIC) and Bayes information criterion (BIC). According to Schreiber et al. ([@CR39]), the smaller these indexes are, the better the model fits the data.

Criterion-related validity {#Sec12}
--------------------------

One of the commonly used approaches in criterion-related validity research is group-comparison. It usually aims to investigate differences between instruments scores across groups of examinees (Goodwin and Leech [@CR14]). A comparison variable should be related to the measuring construct. In this case, we rely on prior studies about MOOC completion. The research indicated that learners with a high level of SRL skills perform better than their lower counterparts (Kizilcec et al. [@CR21]; Milligan et al. [@CR29]). It has been demonstrated that learners, who set goals to finish the course, are less likely to drop out (Handoko et al. [@CR15]).

To assess the criterion-related validity of the OSLQ, we used students' plans to finish the course (the question 'Did you plan to receive the verified certificate at the end of the course?'). We divided students into two groups according to their plans. The first group of students informed us that they planned to receive a verified certificate, the second group did not. Their plan to receive a verified certificate is perceived as the criterion variable. We performed an independent samples t-test to compare mean scores on SRL between these groups.

Results {#Sec13}
=======

Construct-related validity {#Sec14}
--------------------------

Based on the previous research by Barnard et al. ([@CR2]), we tested the original six-factor hierarchical model suggested for the OSLQ. The model was specified according to the following criteria: (1) each item had a nonzero loading on the six first-order factors; (2) standard errors associated with each item were uncorrelated; (3) covariance between six first-order factors was explained by a second-order factor -- self-regulated learning.

Figure [1](#Fig1){ref-type="fig"} provides the standardised path coefficients from the latent variable constructs to the items and from the higher order construct to the latent variable constructs. The values of fit-statistics indicate an unacceptable fit between the model and the observed data, RMSEA = .08 (.08; .09), CFI = .88, and TLI = .86. Each of the fit indexes exceeds the cutoff criteria suggested by Byrne ([@CR8]) and Schreiber et al. ([@CR39]). The results do not support the original six-factor hierarchical model.Fig. 1The six-factor hierarchical model (*N* = 913). Note. χ^2^(164) = 1200.68, *p* = .00; RMSEA = .08 (.08; .09), CFI = .88, and TLI = .86. \*\*\* *p* \< .001.

The six-factor hierarchical model was modified by removing the help-seeking subscale from the model. Relying on the results of the mentioned research suggested that help-seeking behaviour might be not common among MOOC learners, we hypothesise that a five factor hierarchical model could better fit our data. The model was specified according to the following criteria: (1) each item had a nonzero loading on the five first-order factors; (2) standard errors associated with each item were uncorrelated; (3) covariance between five first-order factors was explained by a second-order factor -- self-regulated learning.

The values of fit-statistics indicate an acceptable fit between the five factor hierarchical model and the observed data, RMSEA = .07 (.06; .07), CFI = .94, and TLI = .93. Figure [2](#Fig2){ref-type="fig"} presents the standardised path coefficients from the latent variable constructs to the items and between the constructs. The paths in the second model are all significant (*p* \< .001) with standardised values ranging from .59 to .92 from the first-order factors to the items, and standardised values ranging from .45 to .94 from the second-order factor to the first-order factors.Fig. 2The five factor hierarchical model (*N* = 913). Note.: χ^2^(99) = 513.09, *p* = .00; RMSEA = .07 (.06; .07), CFI = .94, and TLI = .93. \*\*\* *p* \< .001.

Next, we determined which model better fits the data. Table [6](#Tab6){ref-type="table"} provides the fit indexes for each of the models. AIC and BIC for the five-factor hierarchical model are smaller: 53,558.51 \> 42,103.73 and 53,876.41 \> 42,359.02. According to these results, the second model better fits the data. The five-factor hierarchical model gives a better approximation and interpretation of our data about SRL behaviour among MOOC learners. Therefore, we suggest a redefined model of the OSLQ for MOOC learners, because help-seeking skills appear not relevant in this environment.Table 6Comparison of model fit indicesRMSEA (lower bound; upper bound)CFITLIAICBICSix-factor hierarchical model.08 (.08; .09).88.8653,558.5153,876.41Five-factor hierarchical model.07 (.06; .07).94.9342,103.7342,359.02

The average classical reliability (Cronbach's α) ranges from 0.72 to 0.90 for the subscales (Table [7](#Tab7){ref-type="table"}). Mean Cronbach's α for the six-factor model is 0.89, for the five-factor model is 0.88. High reliability coefficients (α \> 0.80) indicate that the questionnaire can be used for both research and diagnostic purposes (Tavakol and Dennick [@CR45]).Table 7Reliability coefficientsGoal settingEnvironment structuringTask strategiesTime managementHelp-seekingSelf-evaluationCronbach's alpha0.830.890.720.730.760.90

Criterion-related validity {#Sec15}
--------------------------

The research shows that successful MOOC learners tend to have higher scores on SRL skills (Milligan et al. [@CR29]; Kizilcec et al. [@CR21]). Since we did not have access to the data about students' grades, we asked them whether they planned to obtain the verified certificate for MOOC completion. Students' answers for the question 'Did you plan to receive the verified certificate at the end of the course?' was used as a comparison variable. The independent samples t-tests for each dimension were conducted to determine if there is a difference in SRL skills between students, which planned to obtain the verified certificate, and which did not (Table [8](#Tab8){ref-type="table"}).Table 8The independent samples t tests resultsGroups95% CI for mean difference*tdfCohen's d*Planned to obtain the verified certificateDid not plan to obtain the verified certificate*NM (SD)NM (SD)*Goal setting36615.31 (.19)54713.81 (.17).99, 1.995.91\*\*\*839.02.39Environment structuring36616.34 (.21)54715.39 (.19).39, 1.493.39\*\*\*839.64.22Task strategies3667.63 (.18)5476.93 (.14).26, 1.143.11\*\*\*768.44.21Time management3668.49 (.18)5477.59 (.14).44, 1.343.92\*\*\*753.33.27Help seeking3669.36 (.23)5478.19 (.17).61, 1.724.12\*\*\*727.57.28Self-evaluation3664.24 (.13)5473.72 (.09).20, .843.14\*\*712.69.22*Note*. \*\*\* *p* \< .001, \*\* *p* \< .01

According to the results, the mean on all of the six OSLQ's subscales statistically differs among the two groups of students. There are statistically significant differences, at .001 or .01 levels of significance, between students who planned to obtain the verified certificate, and those who did not. The results show that students, who planned to receive the certificate, score higher on goal setting, environment structuring, task strategies, time management, help-seeking, and self-evaluation. We find that the difference in average on the OSLQ subscales ranges from 0.21 standard deviations for 'task strategies' to .39 standard deviations for 'goal setting'. Despite the fact that 'help-seeking' dimension might be redundant in MOOCs, it still determines higher educational outcomes.

Discussion and conclusions {#Sec16}
==========================

The main finding of this study is that the original six-factor model for OSLQ is not always effective and the Russian version of the OSLQ has a five-factor hierarchical structure with the second-order factor SRL in the MOOC environment. The results suggest that the OSLQ can be used to measure people's ability to set goals, structure the online environment, manage time, and evaluate their progress in the online environment. However, the subscale 'help-seeking' does not work in the current context of MOOCs.

The poor statistics and logical incoherence of the items from the dimension 'help-seeking' points to one of the main challenges faced by MOOCs -- low communication between MOOCs' students and instructors during the learning process. Researchers demonstrate that 94% of learners do not participate in online discussions (Qiu et al. [@CR34]), and 90% of forum activity is revisiting previous threads (Breslow et al. [@CR7]). In addition, one of the reasons learners withdraw from MOOCs is the failure to understand the content and having no one to turn to for help (Belanger and Thornton [@CR4]). According to Mahasneh et al. ([@CR25]), more than half of students (64%) avoid help-seeking, even if they do not understand the material. The researchers explain students' fear of asking for help as not wanting to be seen as incompetent (Ryan et al. [@CR38]).

The pandemic outbreak in the world, which stopped normal face-to-face classes, shook the whole education system. This suggests online platforms and instructors need to be prepared to embrace this challenge by improving the quality of platforms and developing new pedagogical practices. As educators and researchers state, education is not only a question of good presentation and design, such elements as support and guidance are also essential (Shaughnessy et al. [@CR42]). Eventually, online education has to become an old-new product with different instructor-learner relations and different ways of using information.

Although this study shows important results, it has some limitations. The results were derived from the sample of Russian MOOC learners. As previous studies show, self-regulated learning skills and strategies can be dependent on the environment (Schunk [@CR40]; Barnard et al. [@CR3]) and can function differently in various countries. Therefore, the results may not be generalized to the whole population of MOOC learners. Future research should consider verifying whether the OSLQ works similarly in different cultural contexts. For this purpose, first, comprehensive evidence of the validity of OSLQ in other cultural settings should be found following strict statistical requirements (for example, homogeneous group of respondents, appropriate sample size). Second, Differential Item Functioning (DIF) analysis can be utilised. DIF appears when there are different probabilities of endorsing a given item on a multi-item scale although the measured characteristics are the same (Zumbo [@CR50]).

Further research could analyse convergent validity of the OSLQ through taking two measures that are supposed to be measuring the same construct and show that they are related. In other words, the correlation between OSLQ scores and other instruments, for example, the Academic Self-Regulation Scale (Magno [@CR24]), the Motivated Strategies for Learning Questionnaire (Pintrich et al. [@CR33]) can be investigated. Predictive validity can be assessed through a comparison of the OSLQ scores with a degree of completion of MOOCs taken right after measuring self-regulated learning skills. Finally, investigating heterogeneous samples, not limited to one category of respondents, would show whether our results are valid for other types of MOOC learners.

<https://openedu.ru/>

**Publisher's note**

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The authors express gratitude to Saule Bekova, the Research Fellow of the Centre of Sociology of Higher Education, National Research University Higher School of Economics, who worked as the coordinator of the project "Modern digital educational environment".

This work was supported by the Ministry of Science and Education of the Russian Federation in the framework of the Modern Digital Educational Environment.

The datasets generated and/or analysed during the current study are not publicly available due to the restrictions imposed by the funding organisation but are available from the corresponding author on reasonable request.