# An Information Service to Act in Binomial 'Monitoring-Improvement' of Educational Performance in Portugal: Three Focus Group Studies to Explore the Concept

Antonio Castro, Delfina Soares

► To cite this version:

Antonio Castro, Delfina Soares. An Information Service to Act in Binomial ‘Monitoring–Improvement’ of Educational Performance in Portugal: Three Focus Group Studies to Explore the Concept. IFIP Conference on Information Technology in Educational Management (ITEM) and IFIP Conference on Key Competencies for Educating ICT Professionals (KCICTP), Jul 2014, Potsdam, Germany. pp.278-288, 10.1007/978-3-662-45770-2_24 . hal-01342712

HAL Id: hal-01342712

https://inria.hal.science/hal-01342712

Submitted on 6 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# An Information Service to Act in Binomial ‘Monitoring–
Improvement’ of Educational Performance in Portugal:
Three Focus Group Studies to Explore the Concept

Antonio Castro<fnref n="1" /> and Delfina Soares<fnref n="2" />

**Abstract.** This paper presents an analysis of a focus group discussion, which was carried out to collect information about the need, utility, and value that can be associated with the existence of an information service aimed to support educational leadership activities. The service intends to support leadership in monitoring and improvement of school performance and activities. To check the perceptions of school actors about the existence of this kind of service and to identify perspectives that would add value to the director’s community, three focus group discussions were held in three different Portuguese country regions (Algarve, Lisbon, and Porto). The groups were designed to include a maximum of six school directors that tend to be representative of each region. The duration of group discussions ranged from one hour and ten minutes to one hour and thirty minutes. To stimulate discussion, a roadmap with eleven questions was prepared. The group discussions were audio recorded, transcribed and analysed. This paper details the design and preparation of the focus group activities and presents a reflexive discussion on the data collected.

**Keywords:**. Focus groups, IT education management, educational IT leadership

## 1. Introduction

A huge penetration of Information Technologies (IT) in schools has been witnessed over the last decades. This has produced significant benefits, reported in many scientific studies and technical reports such as ‘Technological modernization of education in Portugal Diagnostic Study’ [1], ‘Technological Modernization of schools’ [2], ‘OECD/CELE review of the secondary school modernization in Portugal’ [3] and ‘The Implementation of the Technological Plan for Education in Portugal, a School Perspective’ [4]. Despite all the benefits pointed out, it is our conviction that currently available IT in schools is not adequately used to its full. In this project, it is advocated that IT can be adequate in terms of supporting bigger benefits, specifically at the monitoring and improvement level of school activities and performance. The existence of an information service could be useful, therefore. A

<footnote n="1">¹Centro ALGORITMI, Department of Information Systems, University of Minho, Portugal
[antonio.castro@algoritmi.uminho.pt](mailto:antonio.castro@algoritmi.uminho.pt)</footnote>

<footnote n="2">²Centro ALGORITMI, Department of Information Systems, University of Minho, Portugal
[dss@dsi.uminho.pt](mailto:dss@dsi.uminho.pt)</footnote>
technologically-based information service that: i) collects; ii) stores; iii) analyses; and
iv) reports data interacting in IT school management ecosystems, in order to produce
new knowledge, may substantially contribute to school performance work. This
research project has been working on the description and specification of such an
information service.

Information services are information technology (IT) based services that perform
functions involving some form of information processing that is of some value in a
market or society sector. The essence of the information service (the concept) that is
explored in this research project encompasses the following aspects: (i) collection (as
automatically as possible) of information from schools and its storage in information
repositories following data warehousing approaches; (ii) processing of this
information, using business intelligence technology, with Key Performance Indicators
(KPIs) in analytical engines; (iii) providing reports and search facilities; (iv)
providing reports and search facilities to support benchmarking analysis to leadership
groups.

* To check the school director’s sensitivity (or any other executive board individual) to the acceptance and functionalities of this kind of information service, three focus groups (FGs) were prepared to discuss these needs. The focus group discussion was selected because it allows interaction among the multiple actors and the emergence of multiple and complementary perspectives [5]. Discussing different perspectives may lead to unexpected findings. Moreover, in terms of research perspectives, the model is open and does not impose any circumscribed opinion, taking richness into the process from unclogging single opinion. This openness of approach allows wide opportunity to comment, expose ideas, and share experiences. Besides providing qualitative and circumscribed information to the research, the use of focus groups also allowed the creation of a set of contacts and connections with interested stakeholders for this research project.

* Three focus groups were conducted. Each focus group involved a maximum of six individuals, all of them with school direction responsibilities. The discussions were located in three different regions of Portugal (Porto, Lisbon and Faro), in March 2013.

## 2. Group Constitution

The process started with a directed and personalised invitation addressed to each school board (the group that directs the business of the school). The invitation sent included: i) a brief description of the research project; ii) the place and date where the discussion would take place; iii) the goals or the focus of the discussion; and iv) a presentation of the research project, its context and a presentation of the research center with some of its work. Every discussion group gained individuals from different places in the region. Up to six confirmations were accepted for each group, representing diversity and heterogeneity from the regions. The event in Algarve happened in Faro, the most representative city in the region, which received school directors from cities of Vila Real Santo António, Albufeira, Portimão and Lagos. The second focus group was run in Lisbon and involved directors from the cities of
Lisbon, Paço de Arços, Algés, Sintra, Peniche and Santarém. The latter focus group took place in Porto and involved directors from Maia, Oliveira do Bairro, and Porto cities.

To make up the groups, there was collaboration from the technological partner in this project. The partnership considers technical issues and access to schools as well. The invitation was sent to school direction boards that at least use software management. Each board agreed to participate, indicating an individual who would be present.

The topics of the discussion were carefully prepared [6] and sequenced. Preparation included experiences gained so far, to cover topics, components and ingredients considered in context. These experiences covered: i) the six Portuguese governmental programmes for IT in education and results to date; ii) the autonomous approach in some schools contracted by the government; iii) the commercial option versus ministerial offer for software management; iv) IT strategic alignment policies to educational projects; and v) Key Performance Indicators used in different domains, such as the Portuguese educational system and the Organisation for Economic Co-operation and Development (OECD). Closing the discussion, a review was shared, to identify and clarify the approach taking and discussed in each matter. The goal and discussion achievement do not look for consensus, in getting collective decisions. Rather, the success of the discussion success was measured through collecting qualitative information arising from participants' opinions. These results were solicited through open questions and procedures where the participants felt absolutely free to choose the form of their response, orally or in mien.

## 3. Session Planning

After the groups had been identified, each session was carefully prepared. They were planned to work through three different sessions: i) open session, to introduce the researcher and the project, ask for permission to record the discussions and give acknowledgment for being present. This session was meant to put the participants at ease, afterward ii) the discussion session, included initially some introduction questions to introduce the elements helping participants to respond freely, some other questions focusing on the key aspects and the kernel of the discussion, and finally some challenges were cast and inducements proposed. Finally, the iii) closure session, during which final acknowledgements were addressed, notes were taken and backups records were made. The conduct of the FG included a pilot phase. It was planned to take about one and a half hours. During this pilot, special attention was given to the quality and sequencing of the questions to be used. It was assumed that answers and individual questions may use an estimated time that would not correspond if answered in-group; a set of eleven questions was defined.

The focus group activities ran in a participant school, equidistant from all participants. There were an invitation to those schools, who all readily agreed.
## 4. The Questionnaire Roadmap

To prepare the discussion, a pilot with questions was designed for each of the different sessions, and some rules were imposed. The first rule to develop the discussion was to establish that dichotomous questions and 'Why' questions were to be avoided, in order to maintain focus. For each theme for discussion, a range of context slides was used to provide support for the discussion and to propitiate and facilitate discussions between the participants. In total, the eleven questions were allocated across the three sessions as follows:

*   **Open questions:** With dual intention, to put participants at ease and develop affinities between group actors. They were asked about professional skills and the length of time in their job. It was intended that there should be a quick answer and it should be done in a factual way, for example - 'director of school ... with x professional years in the job'. Thus, questions were applied like:

**Q1:** Who is the oldest group member? What kind of work have you been doing over this time?

**Q2:** How wide is the gap between the oldest and the youngest school director? What differences can you report happening across that time period?

*   **Introductory questions:** questions that introduce general topics for discussion and provide participants the opportunity to reflect on past experiences and their connections with topics involved. These issues are not critical to analysis, but also were intended to promote the discussion around IT and education management.

**Q3:** With the governmental IT programmes shown in the slide, which one is most prominent in your mind, and what values have these programmes brought to your school?

**Q4:** How was the IT investment leveraged? Both the school and governmental ones?

*   **Transition Questions:** These questions pave the way to the core issue. The questions also drive the participants to the research problem and the research questions.

**Q5:** How can IT contribute to support the educational service?

**Q6:** Is it possible for IT to have more influence in school performance? How?

**Q7:** What has been done and what remains to be done in terms of annual activities planned at this moment?

*   **Key questions:** In this session issues considered were essential to the research. The contribution and acceptance about significance of information services, in terms of additional helpful functionalities, was discussed. Collective reflections answered were encouraged; also in this category analyses of time invested were considered without restrictions. The followed questions were asked:

**Q8:** How is your school positioned now, in terms of ministry goals? If you are on target, how do you measure it?

**Q9:** Are outcomes this academic year aligned with your expected goals for this present year? When deviations occur, how long do these take to be
diagnosed?

**Q10:** Supposing your management software supports the entire school flow process, for what proposes should stored school data be accessible? Should it be shared with other schools? Should it contribute to aggregated information in terms of global knowledge?

**Q11:** Is there any value to leadership and school governance afforded by aggregated information? For example, what benefits can be found when collectives work with other schools? Will there be any advantage in linking leadership and knowledge in between?

* *Final questions:* The questions raised here close the discussion and allow a reflection on materials discussed. These questions remained open and depending on each group were:

Brief questions - If the discussion was considered to have achieved its goals, a brief summary of the essential issues was stated in two or three minutes; then the discussion was closed, highlighting favourable findings and considering: 'Are these the essential matters discussed here? Does someone want to add something else?'

Final question – If the discussion was not considered to have achieved its goals, strategic points were revisited, and discussed briefly before making final acknowledgments to participants and to the host in particular.

## 5. Results

The analytical space was organised to look for ideas that could be of particular interest to the project and to identify key concepts [7]. The elements reported from direct group speech transcriptions were divided into two groups – favourable and unfavourable elements to the service exposed. The key ideas formed or developed through group discussions are listed here without using any priority criteria:

A. Provisioning aggregate information from schoolwork.

B. The facility to build effective school networks

C. Availability and facilitation of electronic process exchange between schools.

D. Performance schoolwork monitoring, individually and in aggregated mode.

E. Providing another intelligence element, a driver to schools' extra activities available in the market.

F. Broad consent to software management through commercial platforms and not as a unique solution to all schools.

These points, retained from discussions as key concepts and ideas, were developed during the discussions, sometimes from discussion, sometimes from an idea launched. To substantiate the description in each idea, some excerpts from the transcripts for each key idea in the discussion follow:

**Idea A:** Provisioning aggregate information from schoolwork

To collect information from schools, and to store it in a huge repository to allow different kinds of aggregate reports, is a base concept from this project. This idea
was not opened in the discussion to explore the concept, but just to ascertain
acceptance. The idea was well accepted, as schools have already been targeted for
deep scrutiny, from different parts of society. Therefore they can discuss
information independently. Most participants showed willingness to take part and
contribute to this experience.

**Idea B:** Building bridges between leaderships to facilitate effective school networks and develop a community of practice

How schools organise relationships between themselves was a matter discussed
but while avoiding undue concern. Building affiliation between schools is difficult
so this facility is seen as useful to this project. There are movements and
associations taking place, but they have a geographical constituency. This element
can add value to the service being developed here; it happened unexpectedly,
emerging the first time in the Algarve group discussion. The matter emerged by an
alternative interpretation of electronic exchange, where an individual in the group
observed and pronounced that there could be further exchange of information and
electronic files, and said "... for example... it also happens with temporary student
exchanges in the programme exchange... - ... our school takes and sends groups
with a German school... I think it is a two year exchange programme... because
they want to learn Portuguese language and it is part of a contract to stay for a
week in Portugal..." Then someone suggested "... broadcasting this event should
be useful..." This idea does not mean the availability of a direct contact between
schools, but a space used by schools as support for exchanges and extra-curricular
activities. A space where stakeholders can share events and activities - from
accommodation to normative assessment, describing the activity itself, the service
can be useful if it has a feature that allows exchange of events and activities
between schools.

**Idea C:** Use the 'service' as a facilitator to student processing for inter-exchange

Facilitating inter-exchange processes in digital format was an idea that gathered
wider support. It was introduced into the discussion in the Algarve group. This idea
received wide acceptance due to procedural volumetric transfer needs concerning
students between schools. Extending the concept, and if possible making it more
comprehensive using the concepts of interoperability, it was an idea that should be
represented further, although the focus in this research project is not to support
operations in and in-between schools, but to connect leadership and knowledge.

**Idea D:** Collecting information to monitor

This key idea is the dominant concept of the information service presented; to
monitor to improve. In discussion groups, the moderator launched the idea not
directly but openly in an attempt to steer the discussion into a confluence of ideas.
Some contributions emerged in discussion.

Initially, the group discussion held in the Algarve was resistant to some
elements, to available information from schools in the form of databases. It was
clear that this point was 'well spoken', for safety in terms of individual rights and
guarantees, need to be ensured by the National Commission for Data Protection
(CNPD). In the Lisbon discussion group, the position was the opposite and indicated willingness to provide details since all issues need to be raised by the CNPD. Some examples from the Lisbon discussion group stated: "... no problem... these are administrative data, we live in a republic, free, and democratic... It is necessary to have guarantees protecting data, which is personal, that must be protected... about the administrative data there is an obligation to school administration to provide them to whom they are entitled... and they do not need to prove they are entitled... or that the constitution that wants to oversee the actions of public administration is calling... I learned this from a judgment about commission access to official documents... by a question released many years ago because a parent asked me... - Please, I would like to know the name, qualifications, professional experience of all teachers of my son... and [he] said to me, look I'm a doctor, I am a surgeon and everyday I make medical surgical interventions, and it does not bother me that someone will ask what is the curriculum... this is a right that is accessible to everyone... therefore the education of my children is no less important than what I do... I was surprised... I made an appointment with the National Commitment Administrative Data, and they sent me the judgment... after... twenty days... with six or seven pages that said... so... everything is nominative, where you live, phone number and so on... I'm not authorised to give it to anyone and I have to manage according to that... Everything that is personal but is not relevant... the situation in their career, the teacher can gain the Masters degree but never gave me anything to put in the biographical record... But his name was entered with the qualification he gained at the Masters level and due to this he has been repositioned in his career... administrative data and the judgment given that ended in the constitution have to provide all those who have an interest in them... and then he wrote... interest is very generic, every citizen has the right to see the acts of the public administration, it is a constitutional right and just tell the citizen to view them... this has to be provided... Therefore more than monitoring it is to investigate to help to enter the market, to create wealth, to help to be more effective and evolve... therefore no one can deny that data."

This matter is delicate and sensitive. There exists some resistance to providing databases even with guarantees. Ensuring CNPD assumptions would enable the gaining of schools' agreements to provide their databases in order to develop a prototype planned in this research project. Because this is a sensitive and critical point, it was always repeated at the end of the discussions, in the review time.

**Idea E:** Availability of another support element, provider of extra-curricular activities to meet different needs. This element should be bi-directional, into and from the 'service'

This idea appeared tenuously in the Algarve group. It was then validated and complimented by other groups. Its acceptance by representative individuals, gave an indication of the strength that this kind of functionality could provide in terms of information and adding value to the service. This element appears to suggest acting as facilitator and a networker between schools and institutions who for some reason maintain a relationship of interest with certain schools in particular and education globally. On the school side there exist interests in all institutions that provide activities that can collaborate and contribute to the construction of
informal learning. From the institution side, there is sometimes difficulty in gaining access to a plurality of schools, across geographic and activity diversity. Thus the validity of an element like this in the 'information service' may act as a 'hub' to approach stakeholders, and make available extra-curricular activities. These activities support construction skills for learners and other ones that may be identified by the elements of interest in the remaining educational community (teachers, aides, parents and administrators).

An illustrative example from the discussions driving this idea, is cited here from an extract from the Algarve group discussion: "... And further down the line, be able to find activities that already have results to help in accomplishing certain goals... taking for example study visits, dedicated activities from universities or pedagogical organisations such physics labs." These were followed soon after by positive reinforcements from other individuals, such as "That should be interesting" and from another individual form Vila Real "Of course... ", or an Albufeira individual "That should be good!... And share examples between schools that have already tried and worked fine." The group argued about the same examples as in this case, with two individuals saying: "Being able to have a list for example... the fifth year, in history discipline... this and this and this... sixth year, in geography discipline... this and this and this..." to complement with a Lagos individual: "but note that the north of the country have more offers with a set of institutions and information that is well crafted.... there is a greater tradition in exchange between institutions and schools."

It was felt that this possibility could provide links to generate social value, well-being and organisational economic performance for the educational establishment and give return to the institutions that provide these activities. This is an element where everyone can gain benefits.

**Idea F:** Broad consent for a school free selection of a software management platform

The concern about selection of software management providers was a point where there was intensive discussion in all groups, mainly on two themes, history on the one hand and selection between commercial providers or the Ministry provider on the other. There was a consensual discussion about the history of software providers. The same did not happen when it was necessary to discuss the option of providing a commercial provider versus a ministerial provider. In historical terms, almost all participants identified their first providers as 'JPMAbreu', 'Prodesis' and 'Truncatura', just naming the most referenced from the last decade. There was also a big concordance about a provider lifecycle up to this time. About providers, two forces dominated the discussions: preferences for commercial providers; and preferences for a unique provider that must be ministerial. The defenders of a unique ministerial provider for management software argue that it would be better for schools essentially for two reasons: i) they do not have any responsibility in that process; and ii) every school has to use the same management software. The defenders of commercial providers for management software argue that they can change when they decide usefully, and it improves competitiveness between providers of management software.

Still on this matter, there was a relevant contribution from the Lisbon focus
group discussion that is important to share. A school director with extensive professional experience said with conviction about the end of the first cycle software that was used led to 'market dry up'. Explained prospectively, software vendors have a period of expansion in license sales, but they are then forced to create structures where programmers become burdens when the new school market has all been served. This justifies their need only to sell 'upgrades', which became unsustainable to the companies, he said. This was also in his view, a strong reason to have a single software vendor. Defenders of commercial options, with some irony and humor, compared the option with the selection of the 'one book' (movement regime of the 1950s and 1960s, which required all schools in the country use the same textbooks).

This matter, for a single ministerial supplier or free appeal to the market, despite not fracturing any group opinion did not lead to consensus. There were different individuals who identified with each of the two options; those who defended the unique solution were in the large minority coinciding with their time in service, and experiences.

Principal elements were identified in focus group discussions. Some may be useful to the project and some might go forward to consider in future work. For the analyses, a second iteration from the discussion records classified strengthening and weakness factors as well. These two characteristics were identified and grouped – favourable elements and unfavourable elements to service viability (defined in Table 1).

<table><caption>**Table 1.** Favourable and unfavourable elements to the exposed service</caption><thead><tr><td>Favourable elements</td><td>Unfavourable Elements</td></tr></thead><tbody><tr><td><b>FAV1:</b> There is no universal system for administrative and pedagogical management, with capacity to collect data from different platforms of the education system, and return this information to schools and stakeholders.</td><td><b>UNF1:</b> Hindrance to cover all school processes, pedagogical and administrative ones.</td></tr><tr><td><b>FAV2:</b> Schools feel great pressure due to the accountability of their work by parents, the media, local and national authorities.</td><td><b>UNF2:</b> Technological pedagogic support adapted to school mission and global measures is missing.</td></tr><tr><td><b>FAV3:</b> Each school produces academic result analyses every trimester.</td><td><b>UNF3:</b> A lack of habit in using the technology in collaborative work.</td></tr><tr><td><b>FAV4:</b> All elements identified an evolution and benefits in the introduction of government programmes for IT in education.</td><td><b>UNF4:</b> Regulatory changes require updates and incur costs.</td></tr><tr><td><b>FAV5:</b> There is a richer technological environment in schools.</td><td><b>UNF5:</b> Evolution and maintenance is absent in platforms used.</td></tr><tr><td><b>FAV6:</b> There is a legal need to produce internal evaluation.</td><td><b>UNF6:</b> Small market.</td></tr><tr><td><b>FAV7:</b> All groups agree positively to</td><td></td></tr></tbody></table>
have a community of practice to share and
connect experiences.

**FAV8:** It is necessary to monitor targets for each educational project, by identifying variances and methods to measure outcomes.

**FAV9:** Wide preferences to choose software management freely according to school needs.

**FAV10:** Lifecycle of IT governmental programmes in education and technologies.

## 6. Reflections and Conclusions

The information service studied in this research project intends to act essentially in two domains: i) aggregate information from diverse schools and classification of information with an analytical engine supported by KPI parameters: and ii) develop a community of practices, connecting leadership groups, knowledge and educational stakeholders.

Performing focus groups with professionals with expertise gave important constructs to consolidate the research. Interacting with school directors, together across the country, conducted in different representative country regions, developed important elements. Some participants expressed their willingness in providing school information and becoming a partner in this research project. This evidenced the advantage that can come through from using focus groups and the interactivity afforded. The acceptance and curiosity aroused by the exposed service and discussion succeeds, generating interest in collaborative work and getting access to independent knowledge.

The use of focus group discussion as a technique was a particularly useful tool, showing useful paths and understanding to fulfil the research need. The discussion allowed school directors' views about evolutionary paths of IT in education and its strategic importance to emerge.

It is also important to refer to limitations found in this process. The moderation of the discussion was done by the researcher, which may have influenced the way the discussion proceeded. The peer interaction led to some inefficiencies in sharing results, led to some difficulties in discussion (essentially when they spoke simultaneously), inefficiencies in individual discussions (discourse drift) taking into discussion irrelevant purposes outside the context, losing precious time and forcing a considerable direction on the discussion. Another difficulty related to the analyses of results. The interaction between group elements in the discussion brought forward comments that deserved to be interpreted. Finally a closing limitation experienced, was advice about group constitution that could be brought to the analyses through different perspectives.

Despite the limitations pointed to in the last paragraph, the decision to undertake focus group discussions proved to be a good option, considering the richness of the
perceptions and opinions shared by the educational stakeholders during the
discussions, and not least gaining a privileged introduction to schools from across the
country. Discussions have led principally to two outcomes: key ideas useful to the
service; and the validation of the service itself, and favourable and unfavourable
aspects identified. The priority was to identify possible ambassadors to the project.
An ambassador needs to be available to participate, providing information and
contributing to refine the project.

The needs for benchmarking, by levels and accessible to school directors, with
independent data from schools, was presented in all discussions. Another relevant
outcome was the need for availability to share school information for such a project.
Other relevant aspects achieved were the availability of building bridges to decrease
distances between schools, direction boards and projects.

It is recognised that the selection of focus group discussion, as a tool to collect and
search for additional information into this research project, is not a perfect tool and
there were some limitation. However, all the discussions were achieved and were
successful, and the focus group discussions demonstrated benefits and were assuredly
a good option to deal with such different sensitivities raised by the different school
directors.

References

1. GEPE: Modernização tecnológica do ensino em Portugal. Estudo de Diagnóstico.
Minitério da Educação, Lisboa, Portugal (2008)

2. Gabinete de Estatística e Planeamento da Educação: Modernização Tecnologica das Escolas. Ministério da Educação, Lisboa, Portugal (2008)

3. Blyth, A., Almeida, R., Forrester, D., Gorey, A., Hostens, G.: Modernising Secondary School Buildings in Portugal. OECD Centre for Effective Learning Environments (CELE), OECD, no place of publication (2009)

4. Castro, A., Santos, L.: Implementation of the Technological Plan for Education in Portugal, a School Perspective. In Tatnall, A., Kereteletswe, O.C., Visscher, A. (eds.) Information Technology and Managing Quality Education. Springer, Boston, MA, 75–85 (2011)

5. Morgan, D. L.: Focus group. Annual Review of Sociology, 22, 129-152. [Retrieved August 31, 2014, from http://www.jstor.org/stable/2083427] (1996)

6. Krueger, R., Casey, M.: Focus Groups: A Practical Guide for Applied Research. Sage Publications, Thousand Oaks, CA (2000)

7. Krueger, R. A.: Analyzing and reporting focus group results. Sage Publications, Thousand Oaks, CA (1998)