# Public Sector IS Maturity Models: Legal Pluralism Invades Public Schools

Helle Zinner Henriksen, Kim Normann Andersen, Rony Medaglia

<figure><img src="image_1.png" /></figure>

## To cite this version:

Helle Zinner Henriksen, Kim Normann Andersen, Rony Medaglia. Public Sector IS Maturity Models: Legal Pluralism Invades Public Schools. 10th Electronic Government (EGOV), Aug 2011, Delft, Netherlands. pp.100-111, 10.1007/978-3-642-22878-0_9. hal-01589089

HAL Id: hal-01589089

https://inria.hal.science/hal-01589089

Submitted on 18 Sep 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Public Sector IS Maturity Models:
Legal Pluralism Invades Public Schools

Helle Zinner Henriksen, Kim Normann Andersen, Rony Medaglia

Department of IT Management
Copenhagen Business School, Howitzvej 60
2000 Frederiksberg, Denmark
{kna.caict, hzh.caict, rm.caict}@cbs.dk

**Abstract:** Online applications and processing of tax forms, driver licenses, and construction permits are examples of where policy attention and research have been united in efforts aiming to categorize the maturity level of e-services. Less attention has been attributed to policy areas with continuous online citizen-public interaction, such as in public education. In this paper we use a revised version of the Public Sector Process Rebuilding (PPR) maturity model for mapping 200 websites of public primary schools in Denmark. Findings reveal a much less favorable picture of the digitization of the Danish public sector compared to the high ranking it has received in the international benchmark studies. This paper aims at closing the gap between the predominant scope of maturity models and the frequency of citizen-public sector interaction, and calls for increased attention to the activities of government where the scale and frequency of the interaction between citizens and government will challenge our concepts of maturity.

**Keywords:** public sector; IS; adoption; maturity models; public schools; primary schools

# 1 Introduction

In 2002, the Danish government published the “Act on transparency and openness in education” (2002 Act), introducing the requirement for public primary schools to use the Internet to provide data regarding student grades and wellbeing. The results of the take up of such measures in public primary schools are indeed very far from the objectives set out in the act. The “360 degree review of schools,” a recently published report, calls for a much stronger focus on results, thus reemphasizing the recommendations from the 2002 Act.

The objective of this paper is to highlight the potential tension between law-driven digitization of public sector activities and adherence to the legal acts. Digitization represents a challenge in relation to the numerous ways of interpreting how the rules and regulations are actually implemented in a satisfactory manner, leading to the
phenomenon of stage models and also to the, by now somehow dated, debate on the digital divide.

These observations have driven this study of how Danish publish schools live up to the objectives of the 2002 Act. Three main issues have formed the basis for this study on public schools' use of the Internet:

- Can grades for each school be accessed simply and quickly on the school website?

- Are assessment criteria and indicators of the achievement of objectives for grade levels available through the website?

- Does the school website provide a comparison of objectives and grades with other schools?

Our tool to analyze the public school websites is the Public Sector Process Rebuilding (PPR) model [1, 2]. The PPR model provides a natural extension of the political statements about openness and comparison mediated through the digital channels. Its essential premise is that the central challenge for the public sector is to organize performance with the user in focus, and to do this with activity content as the focal point in each public agency using ICT.

In the following section we present and discuss a revised version of the PPR model, used as analytical framework to analyze the data of this study. In section 3 we outline the methods used for sampling, data collection and analysis, and highlight some limitations of the study. In section 4 we present and discuss the findings. In the conclusion section we provide a summary of the study results, reflections on the findings, and outline implications for future research on the topic area.

# 2 Analytical Framework

## 2.1. The maturity model as a tool of categorization of e-government

The maturity model we have used to assess school websites is a normative model with four maturity levels (see Figure 1 below). It has as its assumption that the overall goal is to move up to the highest level possible. This assumption is shared with other maturity models for IT in government [3, 4, 5, 6, 7]. It is obviously debatable whether it is a unified and shared objective to pursue the evolutionistic target of ranking on the highest level of a given maturity model [8]. For example, for some schools it can make more sense, from an economical point of view, to remain at a level rather than to rush towards a higher and initially cost-driven level. It is also debatable whether maturity models capture the objectives of being online. For example, Layne and Lee's [3] maturity model has data integration and technological complexity as the two key dimensions; on the basis of these two dimensions, it develops four maturity levels: catalog, transaction, vertical integration and horizontal integration. In comparison with the PPR model and the Gartner Group Open Government Maturity Model [9], the Layne and Lee [3] model is less ambitious in relation to services for citizens because its focus is more technical and organization-oriented.

The PPR maturity model has two dimensions. The horizontal dimension is the activity orientation, while the vertical dimension reflects the user/ customer focus of
the homepages. The maturity level of each website is assessed depending on the extent to which the activities and the clarity of user/ customer are in focus. In comparison with the original PPR model, in this study we rephrase the levels of maturity, and adjust the variables for assessing each website. In addition, we add the dimension “sophistication of IT use” to improve the visualization of using IT as a cornerstone of maturity.

<figure><img src="image_3.png" /><figcaption>Figure 1 – The adjusted PPR maturity model for analysis of the websites in primary schools</figcaption></figure>

The model has four phases of maturity. The first phase (cultivation) is characterized by having no links to key data. In the public school case, this refers to parameters such as marks for grade levels. At this level, there is a long time gap between updating of information on targets and follow-up on previous data; thus websites are designed without a visible focus on user needs. However, this does not mean that public schools in this group are not using IT.

The second phase (extension phase) represents schools with unclear objectives and strategies for the extent to which the Internet should be used, and there is a strong element of reservation towards greater openness about grades. Although general objectives are available and there are links to websites whereby the user can click on data about the school, there are no direct links to the school's data, and users of the website are not encouraged to consult the data. One characteristic of this group of schools is that it seems unclear whether the aim of the website is to create openness, to give as little and as unbiased information as possible, or whether the aim is to keep the user away from pursuing the data. There is thus great risk that the website does not live up to what the digital user wants.
The third phase (customization phase) has unique links to websites where information about grades can be found, as well as links to the school's objectives, and indicators for the retrieval of these are available. There is a solid effort to create credible and visible processes. There are individual user interfaces, and the Internet is used as a key tool to increase transparency. The strategy for these sites seems clear: Internet technology is used for users to easily and simply gain insight into key output targets.

Finally, there is the fourth and final step (sustainability phase). Schools classified at this level use Web 2.0 tools and rich media for dialogue and engagement with current and / or future users. At this stage, it is simple to find data about grade average and variation. There is also data on teachers' job satisfaction and other information that supports the user's assessment of whether schools have clear objectives and whether these are being met. This group of schools is characterized by seeing the Internet not as a means to insulate themselves from users, but as a means of improving transparency and openness - with end users in focus. The four steps can be seen as an expression of legal pluralism.

## 2.2. Legal pluralism

In the field of sociology of law, the concept of legal pluralism caught our attention due to an observable difference in the implementation of the 2002 Act. Legal pluralism focuses on how informal norms shape behavior independently of the explicit rules stated in the law. Legal pluralism has been defined as "the presence in a social field of more than one legal order" [10]. Legal pluralism represents a deviation from the legal centralist conception [11], which is how most non-lawyers perceive law: an exclusive systematic and hierarchical ordering of normative propositions [10]. This view is supported by an acceptance of laws and rules as instruments to maintain a civil society where citizens accept common norms stated in the law [12]. None the less, the acceptance of more than one legal order is well recognized in society where interpretation of legal rules plays a central role. This is best depicted in TV-series such as "Law & Order" where the excitement is the verbal battle of the interpretation of rules and regulation.

The popular court-room dramas get to the root of a dilemma which jurisprudence has been struggling with for centuries: justice. Is justice given e.g. by God or is it socially constructed? Legal pluralism belongs to the latter view “legal centralism would be “a myth, an ideal, a claim, an illusion”, whereas legal pluralism would be the fact” [13]. The view points to law as a means of social control where groups of individuals determine what is socially acceptable within a given range of possible outcomes. Regulation such as the 2002 Act is in other words not a binary code of legal or illegal. Adherence to the Act depends on the expectations of the social group exposed to the publishing of grades. If parents expect grades to be published in a Phase IV format (the sustainability phase) then the school would receive complaints if there were broken links or misleading links represented by Phase I in the maturity model. If the school does not provide the necessary information parents would make use of the commands provided by the 2002 Act.
<figure><img src="image_4.png" /><figcaption>Figure 2 – Four types of legal pluralism in relation to the 2002 Act</figcaption></figure>

Figure 2 shows how we have interpreted the four maturity steps with respect to linking to grade information related to legal pluralism. The school outside the grey area might provide information to parents through their intranet, but does not provide information via public websites. These schools do not live up to the 2002 Act at all. In the grey area, there are four types which, to some extent, live up to the 2002 Act. Schools in the cultivation phase make it difficult for users to find information because the link to the UNI-C national base provides information about all public schools, or no direct link/ guidance for accessing the central UNI-C base. A variation of the cultivation phase is the case where the school web-site provides a broken link. The broken link can reflect that there is infrequent maintenance of the web-site or it can reflect that it has low priority to allocate qualified staff which can secure a robust web-site. The next level, the extension phase, provides a link to the UNI-C base, thus leaving it up to users to find the school in the list of +1,500 public schools included in the UNI-C base. In the customization phase, there is a deep link to specific schools. In the fourth phase, the sustainability phase, the school draws its information on grades from the UNI-C base, thus ensuring that information is automatically updated but also making certain that the data fit into the layout of the overall school website.
## 3 Data Collection

Based on the normative maturity model presented in Figure 1, we have analyzed 200 public school websites. The total number of public primary schools in Denmark is 1,510. The 200 schools analyzed in this study have been selected based on their location. Denmark is divided into five regions that vary in population density. The ratio between population density and number of schools in each region determines the number of schools selected from each region.

The categorization of sites is based on an assessment of the accessibility of the websites. The method is seen as an appropriation of how a real user would use the website. We have lent inspiration from the Human-Computer Interaction discipline and applied the cognitive walkthrough method [14]. Our application of the cognitive walkthrough method means that we perform the assessment of the schools' interface in a task-specific manner (i.e., if the school posts grades on its website), where the assessment is guided by a questionnaire that draws on the parameters in the PPR model.

We have set twelve variables to guide the scoring and assessment of the individual sites. We have used an unweighted distribution of points from the 12 criteria reflecting the degree of user orientation and activity orientation. Each variable has a scale from 1 to 5, with 5 being the highest. The value of zero has been given in those cases where it was not possible to access the variable. The total score thus ranges from 0 to 60 points. The twelve variables are:

- Output such as grade average and variation is clear from the website

- Cost data, satisfaction surveys, etc., centrally located and communicated

- Use of Web 2.0 and rich media for dialogue and mediation

- Unique link to websites where ratios for school outputs can be found

- Customized user interface

- The school's objectives and progress indicators are available

- General data on quality is available

- Specific output indicators of educational objectives for each grade are available, but kept in general terms

- Links to websites with key figures exist, but with strong reservations about this data

- No links to key data quality or quantity of teaching

- Simple websites where the general / average user is in focus

- The content update is not frequent

As a result, each website has been classified in one of the four phases, with 0-24% of the maximum score (60) for placement in the first group, 25-49% of the total score for placement in the second group, 50-74% in the third group, and 75% or more of the maximum score for placement in the last group.

In assessing the 200 individual sites, we have devoted no more than 15 minutes to assess the 12 variables. It is our assumption that additional time for assessment would not be in line with what actual users would expect to require for finding information. If, for instance, there was a link to relevant information and this information was last
updated in 2005, we coded the website to be in phase 2 of the maturity model presented in Fig. 1.

It should also be noted that we did not systematically examine whether there were historical data on the grades available on the websites. Obviously, it would be more interesting for users if there were time series available on the website. It could be questioned whether variables are additive, and whether discrete variables could be considered as a continuum. Both these problems were present in the analysis of the collected data, where a number of schools ranked relatively high on some variables and lower on others. We acknowledge that an unweighted classification is a source of potential errors, leading to an underestimation of the degrees of maturity.

The assessment of websites was meant to represent a broad range of schools by region. We evaluated 28 schools in the North Jutland region, 30 schools in Zealand, 52 in South Denmark, 50 in Mid-Jutland, and 40 in the Capital Region. Table 1 illustrates the number of schools assessed, and the total number of primary schools in the regions. Data on the number of schools can be retrieved from www.nogletal.dk. Despite the attempt to create a stratified selection, there is a minor anomaly in the data, which theoretically could give a biased result. However, it is our assessment that this would not be the case if we had taken the samples outside the selected data population.

<table><caption>Table 1 – Number of primary schools and total number of schools assessed by region</caption><thead><tr><th></th><th>Number of public schools</th><th>Number of schools included</th><th>Average size of school</th><th>Average number of students per class</th></tr></thead><tbody><tr><td>North Jutland Region</td><td>211</td><td>28</td><td>292</td><td>19.1</td></tr><tr><td>Mid-Jutland Region</td><td>381</td><td>50</td><td>358</td><td>20.4</td></tr><tr><td>South Denmark Region</td><td>394</td><td>40</td><td>313</td><td>20.1</td></tr><tr><td>Denmark Capital Region</td><td>299</td><td>30</td><td>514</td><td>21.6</td></tr><tr><td>Zealand Region</td><td>225</td><td>52</td><td>387</td><td>20.4</td></tr><tr><td>TOTAL</td><td>1,510</td><td>200</td><td>372</td><td>20.5</td></tr></tbody></table>

Scores have been given from a general assessment of the websites. We did not make an assessment of website design, but only of its usability and ability in providing openness and transparency. We assumed identical digital user skills and similar levels for digital inclusion among the various schools. We have not investigated any correlations with the size of the school in terms of number of students or budget and use of the Internet. We acknowledge that there could be different resources and prioritization of information on the Internet in larger schools, compared to smaller schools. It should also be noted that there may well be websites with high aesthetic value and high general IT maturity, as well as profound internal digitization, all of which do not necessarily translate into high value in the digital portal and in business focus. The schools that ranked low on our scale could indeed be
good in reaping the benefits of IT in areas other than openness and the transparency of grades given to pupils.

The cognitive walkthrough as research method has recently been criticized for its tendency to focus on surface issues related to the interaction, overlooking more complex behavioral issues in the interaction between user and device [15]. However, for the purpose of this particular study, the objective is precisely to see if and how public schools manage to translate the spirit of the law into a public access of grades, regardless of complexity.

## 4 Main results

In Table 2 we have summarized the main results of the survey of the 200 schools. Nearly two thirds of the schools are placed at maturity phase 1 (66%). Twenty percent of schools (39 schools) are located on phase 2, while a total of 30 schools (15%) are in phases 3 and 4. Seventy-five percent of schools in the first phase have fewer than 5 points.

The overall conclusion is that the majority of the schools are in the cultivation phase. At the opposite end of the scale, in the sustainability phase (step 4), four of the total twelve schools in this category barely match the requirement for this category. None of the 200 schools has received a maximum score.

<table><caption>Table 2 – Main Results: Maturity Levels by number of schools (n,%)</caption><thead><tr><td></td><td>Number of schools</td><td>Percentage distribution</td></tr></thead><tbody><tr><td>Phase 1: Cultivation</td><td>131</td><td>66%</td></tr><tr><td>Phase 2: Extension</td><td>39</td><td>20%</td></tr><tr><td>Phase 3: Customization</td><td>18</td><td>9%</td></tr><tr><td>Phase 4: Sustainability</td><td>12</td><td>6%</td></tr><tr><td>TOTAL</td><td>200</td><td>100%</td></tr></tbody></table>

We have applied the measures in a liberal manner, thus leaving no doubt regarding the benefit of the school. In practice, this means that if there has been uncertainty whether there are clear links to grades, we have consistently coded them as though there were links to grades. The main result is that more than two thirds of the schools do not live up to the goal of openness and transparency on grades online, and could even have been higher if we had used more rigid coding than we did.

In our assessment, 131 of the 200 surveyed schools are located at the absolute bottom stages of the maturity model, namely, the cultivation phase. One example is the Holme School website which contains links to quality reports; however, the links are not active and there is nowhere on the website to find information about the average grades given to the school pupils. The website features a teaching environmental assessment, but this is not commented on, and there are no plans for
how the evaluation is followed up. The Holme School website well represents schools
in the cultivation phase category.

At the second stage of the PPR maturity model, the extension phase, there are 39 schools, approximately 20% of the 200 schools included in our sample. At this stage it is possible to find links to grades granted by the school, but no direct links to the grades within the schools' own website. The user is guided to sites and must then choose from a list of all Danish schools. Typical of this stage is also a widespread reluctance to make even that information available. For example, on the website of the school of Langmark, we found a document questioning the benefits of having student grades available on the web. The school website makes it clear that it is a legal requirement to publish grades online, but it further emphasizes that it is difficult to see how this could be beneficial if published in isolation. In a website page it is stated that the students' "social, personal, professional and democratic competencies represent equal values" – and these are not necessarily reflected in the grades. This statement somehow discourages the parents' interest in monitoring the grades.

A discussion on the appropriateness of publishing grades online falls outside the scope of this paper, but it is obvious that the user of the Langmark school website is not exactly encouraged to search for grades. It should be emphasized that schools at this stage are on a higher scale than what more than two thirds of the schools are, namely, the first step in the maturity model.

Twenty percent of the schools have taken a step up in the maturity model, albeit reluctantly. The Langsø school, for instance, is in this category. The website provides links to the grades of ninth graders, but not for other classes, and does not give access to historical data. There is no link to compare with other schools, and the majority of information on this website has not been updated since last year or longer. The latest information on objectives, for instance, is from 1999.

In the third stage of the model (customization), there are clear and unambiguous links to information about grades for students. There are only 9% of schools in this category. These include the Mariager Fjord School, which has links to grade data in several places on the site, but data appear to remain relatively fragmented. We have found examples outside the selected population of a number of private and independent schools within this stage.

The sustainability phase (stage 4 of the PPR model) clearly provides grade average scores and other figures on the website. The Karise school and the Western High school fall into this category. The two schools located in the Zealand and Jutland regions are examples of extensive information about the grades. One of the schools has the information integrated on the website, and the update of this content is generated using the direct link to the data provider. We could classify only 12 of the 200 schools in this fourth and most advanced stage of the maturity ladder. This corresponds to only 6% of schools. However, four of the schools in this category scored barely enough points to be categorized being at the fourth step.
# 5 Conclusion and further research

Using a revised version of the Public Sector Process Rebuilding (PPR) model [1, 2] for mapping 200 websites for public primary schools, this paper reveals a much less favorable picture of the Danish public sector, as compared to the high ranking received in the international benchmark studies. This study has attempted to answer three overarching questions;

- Can grades for each school be accessed simply and quickly on the school website?

- Are assessment criteria and indicators of the achievement of objectives for grade levels available through the website?

- Does the school website provide a comparison of objectives and grades with other schools?

The answer to these questions is: only to a limited extent. The majority of the schools do not live up to the objectives of transparency and openness with respect to the measures we have studied. Of the 200 schools surveyed, only 15% come up to a level where they have fully implemented the spirit of the law on transparency, comparability and openness over the Internet. The remaining schools (85%) are far behind the goals of the 2002 Act on transparency and openness in education.

The study indicates that it is difficult to see how the law is internalized in the digital management and communication to end-users. There may be two possible reasons for this. Either schools are unaware of how to put information on the Internet, or they are trying to deliberately avoid comparability and accessibility. Whether it is due to ignorance or reluctance, the result is that Danish children and parents in practice do not have access to information about the quality of teaching in the different schools.

In the beginning of this paper we point to legal pluralism, which suggests that we find the explanations in the deliberate avoidance, or at least in the different interpretations, of implementing the law as it was intended. There seems to be acceptance of more than one legal order or interpretation of the law [10]. In this particular case, the shaping of the norm [12], which appears not to create full and user-friendly transparency and openness, can be excused, to some extent, by the complexity of handling the relatively new channel of communication, the Internet. This challenges the perception of successful e-government implementation in Denmark. It is critical because the domain under investigation holds the position of a high degree of scale and frequency of the ongoing interaction between citizens and government. It is further critical because the particular activity of making grades publicly available represents a task that has a relatively low level of discretion as soon as the grade is granted to a student. The discretion is present when the grade is granted to the student; however, as soon as grades from all students are granted, the fulfillment of the 2002 Act is to publish the grades online without further discretion, and does not involve redistribution of resources or individual case assessment.

In many of the websites included in this study it is explicitly argued by the schools that data on grade point average is misleading. It is mentioned that publication of grades can be counterproductive to other objectives of the school, such as welfare, and that the publication of grades may lead to a unilateral focus on those. Furthermore, on several websites it is argued that parents' and pupils' intranets are
already in use, and that the information about grades and student action plans are disseminated through this forum, and thus not visible on the public Internet.

This set of arguments may be subject to further investigations. As far as the use of parental and pupil intranet is concerned, it must be stressed that this platform is closed to students and parents with a direct membership of each class. The parents' and pupils' intranet does not enable future school customers to obtain information. Instead, the publishing and dialogue on grades have to rely on costly physical or asynchronous digital channels that are not necessarily accessible to interested parents who might want to enroll their children in a particular school.

As stated earlier, the PPR model provides a natural extension of the political statements about transparency, openness, and comparison, but has as essential premises not only that the central challenge for the public sector is to organize the performance with the user in focus, but also to do this with content in the activities as the focal point in each school using ICT. In several previous studies, we have found that it appears to be the opposite: IT solutions in the public sector have employees and/ or internal business operations as the focus [16]. Furthermore, it appears that the institutional interests and structures, rather than individual actors and activities, govern the use of IT [17].

This development might be expected, but compared to benefits of objectives, including transparency and openness by means including the Internet, this can act as a limitation to individual teachers. Teachers have particular objectives and goals for their teaching which go beyond high grades and direct measurable outcomes. They may be interested only somewhat in open dialogue mechanisms to achieve their objectives.

The thesis behind the PPR model is such that the litmus test of whether digitization has led to more openness and transparency is not whether one can find information on the web by sacrificing enough time and resources, but rather if the user can simply and quickly find the information. It is a normative model and there is not necessarily an obvious and clear link to the core services of providing quality teaching and high ranking in the maturity model.

The study has not been intended to explain the differences or lack of digital maturity. We acknowledge that in the search for possible causes for the lack of maturity through interviews with school management, it is the teachers and other school employees or other government agencies who would potentially lead to a more varied picture of the individual schools. We aim at conducting such interviews in follow-up research to the data presented in this paper. Also, we acknowledge that we did not do a reverse-link check of links to nationally stored data on the Ministry of Education website. Clearly, an investigation of hyperlinks from the schools' websites to the data stored at the Ministry of Education and the use of the centrally stored data would be another valuable avenue for further research. Finally, we want to call for research attention to the design of maturity models, a fundamental issue that should be addressed with more care when using maturity models to classify multi-faceted institutions, such as public schools. In the ongoing search for maturity models that can cover the areas with most ongoing citizen interaction, attention needs to be given to accounting for the dynamics and transformative elements within each phase.
6 References

1. Andersen, K.: E-government and Public Sector Process Rebuilding, Kluwer, Amsterdam (2004)

2. Andersen, K. V. and Henriksen, H. Z.: E-government maturity models: Extension of the Layne and Lee model, Government Information Quarterly, 23, pp. 236–248 (2006)

3. Layne, K. and Lee, J.W.: Developing Fully Functional E-government: A Four Stage Model, Government Information Quarterly 18, pp. 122-136 (2001)

4. Moon, M.J.: The Evolution of E-government among Municipalities: Rhetoric or Reality? Public Administration Review, 62, pp. 424-433 (2002)

5. Baum, C. and Di Maio, A.: Gartner's four phases of e-government model, Gartner Group Report No. COM-12-6173 (2000)

6. Siau, K. & Long, Y.: Synthesizing egovernment stage models – a meta-synthesis based on meta-ethnography approach. Industrial Management & Data Systems, 105, 4, pp. 443-458 (2005)

7. Dhillon, G. S., Weerakkody, V. & Dwivedi, Y. K.: Realising transformational stage egovernment: a UK local authority perspective. Electronic Government, An International Journal, 5, pp. 162-180 (2008)

8. King, J. L. & Kraemer, K. L.: Evolution and Organizational Information Systems: An Assessment of Nolan's Stage Model. Communications of the ACM, 27, pp. 466-475 (1984)

9. Gartner Group: Gartner Group Open Government Maturity Model. Published by Gartner Group (2010)

10. Griffith, J.: What Is Legal Pluralism, Journal of Legal Pluralism, 24, pp. 1-55 (1986)

11. Hobbes, T.: Leviathan (1651)

12. Durkheim, E.: In Turner, B. S. (ed.) Classical Sociology. Emilie Durkheim on civil society, SAGE Publications, chapter 5, pp. 88-110, (1999)

13. Dupret, B.: Legal Pluralism, Plurality of Laws, and Legal Practices: Theories, Critiques, and Praxiological Respecification, European Journal of Legal Studies, 1, pp. 1-26 (accessed online May 2011)

14. Wharton, C. Bradford, J. Jeffries, J. Franzke, M.: Applying Cognitive Walkthroughs to more Complex User Interfaces: Experiences, Issues and Recommendations CHI '92. pp381–388 (1992)

15. Harrison, M., Campos, J. C. and Loer, K.: Formal analysis of interactive systems: opportunities and weaknesses. In Cairns and Cox (eds.) Research Methods for Human-Computer Interaction, Cambridge University Press. (2008)

16. Andersen, K.N., Henriksen, H. Z., Medaglia, R., Danziger, J. N., ; Sannarnes, M. K. and Enemærke, M.: Fads and Facts of E-Government : A Review of Impacts of E-Government. International Journal of Public Administration. 33, pp. 564-579 (2010)

17. Fountain, J. E.: Building the Virtual State: Information Technology and Institutional Change. Washington, D.C.: Brookings Institution Press (2001)