## 4 Method
### Evaluating the Guide  
To apply the expert evaluation, a real-time Delphi method was adapted [30]. This is a shorter variant of the traditional Delphi method, which takes place during a meeting and uses mechanisms to summarize the answers given immediately. This method anonymizes expert responses, provides controlled feedback, and facilitates iterative consensus-building [31]. The steps proposed to follow the method are defining the coordinator group and expert group, creating a questionnaire for the experts, sharing the guide and the form, collecting data, and analyzing data. For the first step (coordinator group definition), [PERSON] et al. (2012) [30] recommend creating a group of two to six people who are in charge of studying and refining the working protocol (selection and recruitment of experts, timetable, etc. ), studying and approving the list of experts, drawing up questionnaires, encouraging the participation of experts, analyzing the responses of the rounds, and managing the process. In this case, the authors form the coordinator group. The coordinator group carries out the second step (expert group definition). Experts are those who assume the responsibility of issuing judgments and opinions, which is fundamental for the application of the method. The criteria for selection in this case depended on the nature of the topic and the purpose of the study. A traditional approach was considered for their selection (software engineers specialized in the implementation of best practices based on Scrum and DevOps), considering the level of knowledge, experience, publications, and prestige in their field. Additionally, the number of experts also depends on the objectives and budget of each study [30]. In general, it is considered that there should be no fewer than seven experts, and the maximum is around 30. In this case, the target population was 15 experts in the field of software engineering, adoption of agile practices, and process improvement from Latin America who not only have long academic  
Figure 2: Relation among DevOps phase, practice, question, and suggestion. careers in the subject but have also worked in collaboration with companies supporting the implementation of agile practices in software development processes. Once the selection of experts was made, a formal invitation was created through email, outlining the objective of the study, the steps of the method, the questionnaire to be solved, the number of questions to be answered, the time to answer them, the duration of the process, the potential usefulness of the results, and the benefit of their contribution to the research carried out. The coordinator group, responsible for managing the process, did not participate in the expert panel. In the next step, creating the questionnaire, the validation instrument designed by the authors followed the recommendations of [32], which aim to identify key questions on aspects such as clarity in the wording, understanding, and relevance of both the question and the suggestions. Also, the authors evaluated aspects of the usability of the guide associated with its ease of use and navigation [33], in addition to the helpfulness and effectiveness of the suggestions [34]. Usability testing is crucial for an implementation guide, especially for VSEs adopting Scrum and DevOps, as it ensures the guide is easily understood, navigated, and applied despite potential resource limitations and varying expertise levels. By providing direct feedback on the guide's information architecture and practical applicability in real-world scenarios, usability testing helps overcome barriers to implementation, increases adoption and compliance, reduces errors and rework, and validates expert feedback, leading to improved processes and business outcomes by creating a usable resource [35]. The coordinator group carried out this.