MATHEMATICAL LITERACY FOR LIVING FROM OECD-PISAPERSPECTIVE
Jan de Lange
Freudenthal Institute, Utrecht University – the Netherlands
The OECD/PISA study is spreading all over the globe: 58 countries might participate
in 2006. And although PISA has had ample media attention it seems far from clear
what PISA actually measures and how. The Literacy aspect is often overlooked, andthe discussion is hardly ever about the content, or the instrument. This is anundesirable situation. Given the expansion of PISA it is worth to reflect on its meaning,
possibilities and problems, starting with the Mathematical Literacy aspect.
MATHEMATICAL LITERACY
Mathematical Literacy has become a rather common term through the influence from
OECD/PISA. But there is quite a history, at least dating back to the seventies, of efforts
to cover the idea, often called numeracy or quantitative literacy. As quantitative
Literacy has been used rather widely let us first look at a definition of QL. Lynn Arthur
Steen (2001) pointed out that there are small but important differences in the several
existing definitions and, although he did not suggest the phrase as a definition, referred
to QL as the ‘capacity to deal effectively with the quantitative aspects of life.’ Indeed,most existing definitions Steen mentioned give explicit attention to number, arithmetic,
and quantitative situations, either in a rather narrow way as in the National Adult
Literacy Survey (NCES, 1993):
The knowledge and skills required in applying arithmetic operations, either alone orsequentially, using numbers embedded in printed material (e.g., balancing a check book,completing an order form).

or more broadly as in the International Life Skills Survey (ILSS, 2000):
An aggregate of skills, knowledge, beliefs, dispositions, habits of mind, communication
capabilities, and problem solving skills that people need in order to engage effectively inquantitative situations arising in life and work.

The problem we have with these definitions is their apparent emphasis on quantity.
Mathematical literacy is not restricted to the ability to apply quantitative aspects ofmathematics but involves knowledge of mathematics in the broadest sense. As an
example, being a foreigner who travels a great deal in the United States, I often ask
directions of total strangers. What strikes me in their replies is that people are generally
very poor in navigation skills: a realization of where you are, both in a relative andabsolute sense. Such skills include map reading and interpretation, spatial awareness,‘grasping space’ (Freudenthal, 1973), understanding great circle routes, understandingplans of a new house, and so on. All kinds of visualization belong as well to the literacy

aspect of mathematics and constitute an absolutely essential component for literacy, as
the three books of Tufte (1983, 1990, 1997) have shown in a very convincing way.
We believe that describing what constitutes mathematical literacy necessitates not only
this broader definition but also attention to changes within other school disciplines.
The Organization for Economic Cooperation and Development (OECD) publication
Measuring Student Knowledge and Skills (1999) presents as part of reading literacy alist of types of texts, the understanding of which in part determines what constitutes
literacy. This list comes close, in the narrower sense, to describing many aspects ofquantitative literacy. The publication mentions, as examples, texts in various formats:Forms: tax forms, immigration forms, visa forms, application forms,
questionnaires;
Information sheets: timetables, price lists, catalogues, programs;
Vouchers: tickets, invoices;
Certificates: diplomas, contracts;
Calls and advertisements;
Charts and graphs; iconic representations of data;
Diagrams;
Tables and matrices;
Lists;Maps.
The definition Steen used in Mathematics and Democracy: The Case for Quantitative
Literacy (2001) refers to these as ‘document literacy’, following a definition adopted
by the National Center for Education Statistics (1993).
Against this background of varying perspectives, we adapt for ‘mathematical literacy’a definition that is broad but also rather ‘mathematical’:Mathematics literacy is an individual’s capacity to identify and understand the role thatmathematics plays in the world, to make well-founded judgments, and to engage inmathematics in ways that meet the needs of that individual’s current and future life as aconstructive, concerned and reflective citizen (OECD, 1999).

This definition was developed by the Expert Group for Mathematics of the Programme
for International Student Assessment (PISA), of which the author is chair.
A MATTER OF DEFINITIONSHaving set the context, it seems appropriate now to make clear distinctions among
types of literacies so that, at least in this essay, we do not declare things equal that are
not equal. For instance, some equate numeracy with quantitative literacy; others equate
quantitative and mathematical literacy. To make our definitions functional, we connect
them to our phenomenological categories:
Spatial Literacy. We start with the simplest and most neglected: spatial literacy.
Spatial literacy supports our understanding of the (three-dimensional) world in which
we live and move. To deal with what surrounds us, we must understand properties ofobjects, the relative positions of objects and the effect thereof on our visual perception,
the creation of all kinds of two- and three-dimensional paths and routes, navigational
practices, shadows – even the art of Escher.
Numeracy. The next obvious literacy is numeracy, fitting as it does directly intoquantity. We can follow, for instance, Treffers’ (1991) definition, which stresses theability to handle numbers and data and to evaluate statements regarding problems andsituations that invite mental processing and estimating in real-world contexts.
Quantitative Literacy. When we look at quantitative literacy, we are actually looking at
literacy dealing with a cluster of phenomenological categories: quantity, change andrelationships, and uncertainty. These categories stress understanding of, andmathematical abilities concerned with, certainties (quantity), uncertainties (quantity as
well as uncertainty), and relations (types of, recognition of, changes in, and reasons for
those changes).
Mathematical Literacy. We think of mathematical literacy as the overarching literacy
comprising all others. Thus we can make a visual representation as follows:

Fig. 1: Tree structure mathematical literacy
PISA LITERACYThe OECD/PISA mathematical literacy domain is concerned with the capacities ofstudents to analyse, reason, and communicate ideas effectively as they pose, formulate,solve and interpret mathematics in a variety of situations. The assessment focuses on
real-world problems, moving beyond the kinds of situations and problems typicallyencountered in school classrooms. In real-world settings, citizens regularly facesituations when shopping, travelling, cooking, dealing with personal finances, et cetera,in which mathematical competencies would be of some help in clarifying or solving aproblem.

Citizens in every country are increasingly confronted with a myriad of issues involving
quantitative, spatial, probabilistic or relational reasoning. The media are full of
information that use and misuse tables, charts, graphs and other visual representations
to explain or clarify matters regarding the weather, economics, medicine, sports,
environment, to name a few. Even closer to the daily life of every citizen are skills
involving reading and interpreting bus or train schedules, understanding energy bills,
arranging finances at the bank, economising resources, and making good businessdecisions, whether it is bartering or finding the best buy. Thus, literacy in mathematics
is about the functionality of the mathematics you have learned at school. Thisfunctionality is important for students to survive in a successful way in the present
information and knowledge society.
Such uses of mathematics are based on skills learned and practiced through the kindsof problems that typically appear in textbooks and classrooms. However, they demand
the ability to apply those skills in a less structured context, where the directions are notso clear. People have to make decisions about what knowledge may be relevant, what
process will lead to a possible solution, and how to reflect on the correctness of theanswer found.
Some explanatory remarks are in order for the definition of mathematical literacy to
become transparent:1.

In using the term ‘literacy’, we want to emphasize that mathematicalknowledge and skills that have been defined and are definable within thecontext of a mathematics curriculum, do not constitute our primary focus here.
Instead, what we have in mind is mathematical knowledge put into functionaluse in a multitude of contexts in varied, reflective, and insight-based ways.

2.

Mathematical literacy cannot be reduced to – but certainly presupposes –knowledge of mathematical terminology, facts, and procedures as well as
numerous skills in performing certain operations, carrying out certain
methods, and so forth. Also, we want to emphasize that the term ‘literacy’ is
not confined to indicating a basic, minimum level of functionality only. On
the contrary, we think of literacy as a continuous, multidimensional spectrum
ranging from aspects of basic functionality to high-level mastery.

3.

A crucial capacity implied by our notion of mathematical literacy is the ability
to pose, formulate and solve intra- and extra-mathematical problems within avariety of domains and settings. These range from purely mathematical ones
to ones in which no mathematical structure is present from the outset but may
be successfully introduced by the problem poser, problem solver, or both.

4.

Attitudes and emotions (e.g., self-confidence, curiosity, feelings of interest
and relevance, desire to do or understand things) are not components of thedefinition of mathematical literacy. Nevertheless they are important
prerequisites for it. In principle it is possible to possess mathematical literacy
without possessing such attitudes and emotions at the same time. In practice,however, it is not likely that such literacy will be exerted and put into practiceby someone who does not have some degree of self-confidence, curiosity,
feeling of interest and relevance, and desire to do or understand things that
contain mathematical components.
MATHEMATISATION AS KEY TO LITERACY
Mathematical literacy is about dealing with ‘real’ problems. That means that these
problems are not ‘purely’ mathematical but are placed in some kind of a ‘situation’. In
short, the students have to ‘solve’ a real world problem requiring to use the skills andcompetencies they have acquired trough schooling and life experiences. A
fundamental role in that process is referred to as ‘mathematisation’.
The process of mathematisation starts with a problem situated in reality (1). Next the
problem solver tries to identify the relevant mathematics, and reorganizes the problem
according to the mathematical concepts identified (2) followed by gradually trimmingaway the reality (3). These three steps lead us from a real world problem to amathematical problem. The fourth step may not come as a surprise: solving themathematical problem (4). Now the question arises: what is the meaning of this strictly
mathematical solution in terms of the real world (5)?
The following figure shows the cyclic character of the mathematisation process:Fig. 2: The mathematisation cycle
In might help the interested reader understanding this cycle better by giving an
example. The following item has been field trialled for PISA but was rejected becauseof the high degree of difficulty. The item has previously been released by OECD. For
the purpose of making the mathematisation process clear we have changed thequestion asked:

HEARTBEAT

For health reasons people should limit their efforts, for instance during sports, in order not to exceed acertain heartbeat frequency.
For years the relationship between a person’s recommended maximum heart rate and the person’s age wasdescribed by the following formula:
Recommended maximum heart rate = 220 – age
Recent research showed that this formula should be modified slightly. The new formula is as follows:
Recommended maximum heart rate = 208 – (0.7 × age)

QUESTION 1: HEARTBEAT
What is the main difference between the two formulas and how do they affect the maximum allowable
heart rate?

Fig. 3: Example of item that lends itself to mathematisation
The process of mathematisation starts with a problem situated in reality (1).
As will be clear from the item the realty in this case is physical fitness. An important
rule when doing exercises is that one should not go to far in becoming fit, as this may
cause health problems. We can jump to this conclusion because of the text: maximal
heartbeat.
Next the problem solver tries to identify the relevant mathematics, and reorganizes the
problem according to the mathematical concepts identified (2).
It seems clear that we have two word formulas that need to be understood, and we arerequested to compare these two formulas, and what they really mean in mathematicalterms. The formulas give a relation between the advised maximum heart beat rate andthe age of a person.
Next we are gradually trimming away the reality (3).
There are different ways to move the problem to a strictly mathematical problem, or
trimming away reality. One way to go is to make the word formulas into more formal
expressions like:y = 220 – x
y = 208 – 0.7x
Of course we have to remind ourselves that y expresses the maximum heart beat in b/m
and x the age in years. Another way to go to a ‘strictly’ mathematical world is drawing
the graphs right from the word formulas.
These three steps lead us from a real world problem to a mathematical problem.

The fourth step may not come as a surprise: solving the mathematical problem (4).
The mathematical problem at hand is to compare the two formulas or graphs, and tosay something about the differences for people of a certain age. A nice way to start is tofind out where the two formulas give equal results or where the two graphs intersect.
So we can solve this by solving the equation:
220 – x = 208 – 0,7x.
This give us x = 40 and the corresponding value for y is 180. So the two graphsintersect at the point (40, 180).
And as the slope of the first formula is -1, and of the second one -0,7 we know that thesecond graph is ‘less steep’ than the first one. Or the graph of y – 220 – x lies ‘above’
the graph of y = 208 – 0,7x for values of x smaller than 40, and below for values of xlarger than 40.
Now let us move to step 5 to see if this solution of the mathematical problem helps us
in solving the ‘real world’ problem.
Now the question arises: what is the meaning of this strictly mathematical solution interms of the real world (5)?
The meaning is not too difficult if we realize that x is the age of a person and y the
maximum heart beat. If one is 40 years old both formulas give the same result: amaximum heart beat of 180. The ‘old’ rule allows for higher heart rates for younger
people: in the extreme, if the age is zero the maximum is 220 in the old formula andonly 208 in the new formula.
But for older people, in this case for those over 40, the more recent insights allow for ahigher maximum heartbeat. As an example, and again extreme: for an age of 100 years
we see that the old formula give us a maximum of 120 and the new one 138.
Of course we have to realize a number of other things: the formulas lack mathematicalprecision and give a feel of quasi scientific. It is only a rule of thump that should be
used with caution. An other point is that for ages at the extreme the outcomes should betaken with even more hesitation.
What this example shows is that even with relatively ‘simple’ items, in the sense thatthey can be used within the restrictions of a large international study and can be solvedin a short time; we still can identify the full cycle of mathematisation and problem
solving.
What the example also shows is that mathematical literacy goes beyond mostcurricular mathematics. We will explore this further next.
MATHEMATICS FOR LITERACY
Mathematics school curricula are organized into strands that classify mathematics as astrictly compartmentalized discipline with an over-emphasis on computation andformulas. This organization makes it almost impossible for students to see
mathematics as a continuously growing scientific field that continually spreads into

new fields and applications. Students are not positioned to see overarching concepts
and relations, so mathematics appears to be a collection of fragmented pieces of factual
knowledge.
‘What is mathematics?’ is not a simple question to answer. A person asked at random
will most likely answer ‘Mathematics is the study of number.’ Or, if you’re lucky,‘Mathematics is the science of number.’ And, as Devlin (1994) states in his very
successful book Mathematics: The Science of Patterns, the former is a huge
misconception based on a description of mathematics that ceased to be accurate some
2,500 years ago. Present-day mathematics is a thriving, worldwide activity; it is an
essential tool for many other domains like banking, engineering, manufacturing,
medicine, social science, and physics. The explosion of mathematical activity that has
taken place in the twentieth century has been dramatic. At the turn of the nineteenth
century, mathematics could reasonably be regarded as consisting of about 12 distinct
subjects: arithmetic, geometry, algebra, calculus, topology and so on. The similarity
between this list and the present-day school curricula list is amazing.
A more reasonable figure for today, however, would be between 60 and 70 distinct
subjects. Some subjects (e.g., algebra, topology) have split into various sub fields;
others (e.g., complexity theory, dynamical systems theory) are completely new areasof study.
Mathematics could be seen as the language that describes patterns – both patterns innature and patterns invented by the human mind. Those patterns can either be real or
imagined, visual or mental, static or dynamic, qualitative or quantitative, purely
utilitarian or of little more than recreational interest. They can arise from the world
around us, from depth of space and time, or from the inner workings of the humanmind.
Since the goal of OECD/PISA is to assess student’s capacity to solve real problems, thestrategy has been to define the range of content that will be assessed using aphenomenological approach to describing the mathematical concepts, structures orideas. This means describing content in relation to the phenomena and the kinds ofproblems for which it was created. This approach ensures a focus in the assessment thatis consistent with domain definition, but covers a range of content that includes what is
typically found in other mathematics assessments and in national mathematics
curricula.
The concept of a phenomenological organization is not new. In 1990, theMathematical Sciences Education Board published On the Shoulders of Giants: New
Approaches to Numeracy, a book that made a strong plea for educators to help students
delve deeper to find the concepts that underlie all mathematics and thereby better
understand the significance of these concepts in the world. The labelling ofphenomenological categories has many varieties. For the PISA study the term
‘overarching idea’ has been chosen.

Many phenomenological categories can be identified and described. In fact the domainof mathematics is so rich and varied that it would not be possible to identify an
exhaustive list of phenomenological categories. It is important for purposes ofassessment, however, for any selection of big ideas that is offered to represent a
sufficient variety and depth to reveal the essentials of mathematics and their relations
to the traditional strands.
The following list of mathematical phenomenological categories meets thisrequirement:
Quantity
Space and shape
Change and relationships
Uncertainty
Using these four categories, mathematics content can be organized into a sufficientnumber of areas to help ensure a spread of items across the curriculum, but also a smallenough number to avoid an excessively fine division – which would work against afocus on problems based in real-life situations. Each phenomenological category is anencompassing set of phenomena and concepts that make sense together and may be
encountered within and across a multitude of quite different situations. By their very
nature, each idea can be perceived as a general notion dealing with a generalized
content dimension. This implies that the categories or ideas cannot be sharply
delineated vis-à-vis one another. Rather, each represents a certain perspective, or pointof view, which can be thought of as possessing a core, a centre of gravity, and asomewhat blurred penumbra that allow intersection with other ideas. In principle, any
idea can intersect with any other idea. (For a more detailed description of these four
categories or ideas, please refer to the PISA framework (OECD, 2002).)
QuantityThis overarching idea focuses on the need for quantification to organize the world.
Important aspects include an understanding of relative size, recognition of numericalpatterns, and the ability to use numbers to represent quantifiable attributes ofreal-world objects (measures). Furthermore, quantity deals with the processing andunderstanding of numbers that are represented to us in various ways. An important
aspect of dealing with quantity is quantitative reasoning, whose essential components
are developing and using number sense, representing numbers in various ways,
understanding the meaning of operations, having a feel for the magnitude of numbers,
writing and understanding mathematically elegant computations, doing mentalarithmetic, and estimating.
Space and ShapePatterns are encountered everywhere around us: in spoken words, music, video, traffic,
architecture, and art. Shapes can be regarded as patterns: houses, office buildings,
bridges, starfish, snowflakes, town plans, cloverleaf, crystals, and shadows. Geometricpatterns can serve as relatively simple models of many kinds of phenomena, and theirstudy is desirable at all levels (Grünbaum, 1985). In the study of shapes andconstructions, we look for similarities and differences as we analyze the components ofform and recognize shapes in different representations and different dimensions. The
study of shapes is closely connected to the concept of ‘grasping space’ (Freudenthal,1973) – learning to know, explore, and conquer, in order to live, breathe, and movewith more understanding in the space in which we live. To achieve this, we must beable to understand the properties of objects and the relative positions of objects; wemust be aware of how we see things and why we see them as we do; and we must learn
to navigate through space and through constructions and shapes. This requires
understanding the relationship between shapes and images (or visual representations)such as that between a real city and photographs and maps of the same city. It also
includes understanding how three-dimensional objects can be represented in two
dimensions, how shadows are formed and interpreted, and what perspective is and how
it functions.
Change and Relationships
Every natural phenomenon is a manifestation of change, and in the world around us amultitude of temporary and permanent relationships among phenomena are observed:
organisms changing as they grow, the cycle of seasons, the ebb and flow of tides,
cycles of unemployment, weather changes, stock exchange fluctuations. Some of thesechange processes can be modelled by straightforward mathematical functions: linear,
exponential, periodic or logistic, discrete or continuous. But many relationships fall
into different categories, and data analysis is often essential to determine the kind of
relationship present. Mathematical relationships often take the shape of equations orinequalities, but relations of a more general nature (e.g., equivalence, divisibility) may
appear as well. Functional thinking – that is, thinking in terms of and about
relationships – is one of the fundamental disciplinary aims of the teaching ofmathematics. Relationships can take a variety of different representations, including
symbolic, algebraic, graphic, tabular, and geometric. As a result, translation between
representations is often of key importance in dealing with mathematical situations.
Uncertainty
Our information-driven society offers an abundance of data, often presented as
accurate and scientific and with a degree of certainty. But in daily life we areconfronted with uncertain election results, collapsing bridges, stock market crashes,
unreliable weather forecasts, poor predictions of population growth, economic models
that do not align, and many other demonstrations of the uncertainty of our world.
Uncertainty is intended to suggest two related topics: data and chance, phenomena thatare the subject of mathematical study in statistics and probability, respectively. Recent
recommendations concerning school curricula are unanimous in suggesting thatstatistics and probability should occupy a much more prominent place than they havein the past (Cockroft, 1982; LOGSE, 1990; MSEB, 1993; NCTM, 1989, 2000).
Specific mathematical concepts and activities that are important in this area include

collecting data, data analysis, data display and visualization, probability, andinference.
MATHEMATICS VERSUS MATHEMATICAL LITERACY
In an interview in Mathematics and Democracy, Peter T. Ewell (2001) was asked:“The Case for Quantitative Literacy argues that quantitative literacy is not merely aeuphemism for mathematics but is something significantly different – less formal andmore intuitive, less abstract and more contextual, less symbolic and more concrete. Is
this a legitimate and helpful distinction?” Ewell answered that indeed this distinction
is meaningful and powerful.
The answer to this question depends in large part on the interpretation of what
constitutes good mathematics. We can guess that in Ewell’s perception, mathematics isformal, abstract, and symbolic – a picture of mathematics still widely held. Ewell
continued to say that literacy implies an integrated ability to function seamlessly
within a given community of practice. Functionality is surely a key point, both in itself
and in relation to a community of practice, which includes the community ofmathematicians. Focusing on functionality gives us better opportunity to bridge gaps
or identify overlaps. In the same volume, Alan H. Schoenfeld (2001) observed that in
the past, literacy and what is learned in mathematics classes were largely disjointed.
Now, however, they should be thought of as largely overlapping and taught as largely
overlapping. In this approach, which takes into consideration the changing perceptionof what constitutes mathematics, mathematics and mathematical literacy are positively
not disjointed.
For Schoenfeld, the distinction most likely lies in the fact that as a student he neverencountered problem-solving situations, that he studied only ‘pure’ mathematics and,
finally, that he never saw or worked with real data. Each of these is absolutely essential
for literate citizenship, but none even hints at defining what mathematics is needed for
ML, at least not in the traditional school mathematics curricula descriptions ofarithmetic, algebra, geometry, and so on.
Again, in Mathematics and Democracy, Wade Ellis, Jr. (2001) observes that many
algebra teachers provide instruction that constricts rather than expands student
thinking. He discovered that students leaving an elementary algebra course could solve
fewer real-world problems after the course than before it: after completing the course,
they thought that they had to use symbols to solve problems they had previously solved
using only simple reasoning and arithmetic. It may come as no surprise that Ellis
promotes a new kind of common sense – a quantitative common sense based onmathematical concepts, skills, and know-how. Despite their differences, however,Schoenfeld and Ellis seem to share Treffers’ (1991) observation that innumeracy might
be caused by a flaw in the structural design of instruction.
These several observers seem to agree that in comparison with traditional schoolmathematics, ML is less formal and more intuitive, less abstract and more contextual,less symbolic and more concrete. ML also focuses more attention and emphasis on
reasoning, thinking, and interpreting as well as on other very mathematicalcompetencies. To get a better picture of what is involved in this distinction, we firstneed to describe what Steen (2001) called the ‘elements’ needed for ML. With aworking definition of ML and an understanding of the elements (or ‘competencies’ as
they are described in the PISA framework) needed for ML, we might come closer toanswering our original question – what mathematics is important? – or formulating abetter one.
COMPETENCIES NEEDED FOR MATHEMATICAL LITERACY
The competencies that form the heart of the ML description in OECD/PISA seem, formost parts, in line with the so called ‘elements’ in Steen (2001). The content part of thecompetency description relies on the work of Niss (1999) and his Danish colleagues,
but similar formulations can be found in the work of many others representing many
countries (as indicated by Neubrand et al., 2001). The same holds for the description of
the competency clusters. The present description is a further development of work
carried out by the author (Boertien and De Lange, 1994; De Lange, 1995, 1999) and hiscolleagues from the Netherlands and the United States.
1.

Mathematical thinking and reasoning. Posing questions characteristic ofmathematics; knowing the kind of answers that mathematics offers,
distinguishing among different kinds of statements; understanding andhandling the extent and limits of mathematical concepts.

2.

Mathematical argumentation. Knowing what proofs are; knowing how proofs
differ from other forms of mathematical reasoning; following and assessing
chains of arguments; having a feel for heuristics; creating and expressing
mathematical arguments.

3.

Mathematical communication. Expressing oneself in a variety of ways in oral,
written, and other visual form; understanding someone else’s work.

4.

Modelling. Structuring the field to be modelled; translating reality into
mathematical structures; interpreting mathematical models in terms ofcontext or reality; working with models; validating models; reflecting,analyzing, and offering critiques of models or solutions; reflecting on themodelling process.

5.

Problem posing and solving. Posing, formulating, defining, and solving
problems in a variety of ways.

6.

Representation. Decoding, encoding, translating, distinguishing between, andinterpreting different forms of representations of mathematical objects andsituations as well as understanding the relationship among different
representations.

7.

Symbols. Using symbolic, formal, and technical language and operations.

8.

Tools and technology. Using aids and tools, including technology whenappropriate.

To be mathematically literate, individuals need all these competencies to varying
degrees, but they also need confidence in their own ability to use mathematics andcomfort with quantitative ideas. An appreciation of mathematics from historical,
philosophical, and societal points of view is also desirable.
It should be clear from this description why we have included functionality within themathematician’s practice. We also note that to function well as a mathematician, aperson needs to be literate. It is not uncommon that someone familiar with amathematical tool fails to recognize its usefulness in a real-life situation (Steen, 2001).
Neither is it uncommon for a mathematician to be unable to use common-sense
reasoning (as distinct from the reasoning involved in a mathematical proof).
As Deborah Hughes-Hallett (2001) made clear in her contribution to Mathematics and
Democracy, one of the reasons that ML is hard to acquire and hard to teach is that it
involves insight as well as algorithms. Some algorithms are of course necessary: it isdifficult to do much analysis without knowing arithmetic, for example. But learning(or memorizing) algorithms is not enough: insight is an essential component ofmathematical understanding. Such insight, Hughes-Hallett noted, connotes anunderstanding of quantitative relationships and the ability to identify thoserelationships in an unfamiliar context; its acquisition involves reflection, judgment,
and above all, experience. Yet current school curricula seldom emphasize insight anddo little to actively support its development at any level. This is very unfortunate. The
development of insight into mathematics should be actively supported, starting before
children enter school.
Many countries have begun to take quite seriously the problems associated withoveremphasizing algorithms and neglecting insight. For example, the Netherlands has
had some limited success in trying to reform how mathematics is taught. To outsiders,
the relatively high scores on the Third International Mathematics and Science Study(TIMSS) and TIMSS-R by students in the Netherlands appear to prove this, but theresults of the Netherlands in the PISA study should provide even more proof.
COMPETENCY CLUSTERS
We do not propose development of test items that assess the above skills individually.
When doing real mathematics, it is necessary to draw simultaneously upon many of
those skills. In order to operationalise these mathematical competencies, it is helpful toorganize the skills into three clusters:
Reproduction (definitions, computations).
Connections (and integration for problem solving).
Reflection, (mathematical thinking, generalization, and insight).
We will elaborate on these clusters next.

Cluster 1. Reproduction
At this first cluster, we deal with the matter dealt with in many standardized tests, aswell in comparative international studies, and operationalised mainly inmultiple-choice format. It deals with knowledge of facts, representing, recognizing
equivalents, recalling mathematical objects and properties, performing routine
procedures, applying standard algorithms, and developing technical skills. Dealing andoperating with statements and expressions that contain symbols and formulas in‘standard’ form also relate to this level.
Items at cluster 1 are often in multiple-choice, fill-in-the-blank, matching, or(restricted) open-ended questions format.
Cluster 2. Connections
At this cluster we start making connections between the different strands and domains
in mathematics and integrate information in order to solve simple problems in which
students have a choice of strategies and a choice in their use of mathematical tools.
Although the problems are supposedly non-routine, they require relatively minor
mathematisation. Students at this level are also expected to handle different forms ofrepresentation according to situation and purpose. The connections aspect requires
students to be able to distinguish and relate different statements such as definitions,claims, examples, conditioned assertions, and proof.
From the point of view of mathematical language, another aspect at this cluster isdecoding and interpreting symbolic and formal language and understanding its
relations to natural language. Items at this cluster are often placed within a context andengage students in mathematical decision making.
Cluster 3. Reflection
At cluster 3, students are asked to mathematise situations (recognize and extract themathematics embedded in the situation and use mathematics to solve the problem).
They must analyse, interpret, develop their own models and strategies, and makemathematical arguments including proofs and generalizations. These competencies
include a critical component and analysis of the model and reflection on the process.
Students should not only be able to solve problems but also to pose problems.
These competencies function well only if the students are able to communicate
properly in different ways (e.g., orally, in written form, using visualizations).
Communication is meant to be a two-way process: students should also be able tounderstand communication with a mathematical component by others. Finally wewould like to stress that students also need insight competencies – insight into thenature of mathematics as a science (including the cultural and historical aspect) andunderstanding of the use of mathematics in other subjects as brought about throughmathematical modelling.
As is evident, the competencies at cluster 3 quite often incorporate skills andcompetencies usually associated with the other two clusters. We note that the wholeexercise of defining the three clusters is a somewhat arbitrary activity: There is no clear
distinction between different clusters, and both higher- and lower-level skills andcompetencies often play out at different clusters.
Cluster 3, which goes to the heart of mathematics and mathematical literacy, is difficult
to test. Multiple-choice is definitely not the format of choice at cluster 3. Extended
response questions with multiple answers (with or without increasing level ofcomplexity) are more likely to be promising formats. But both the design and thejudgment of student answers are very, if not extremely, difficult.
It goes without saying that we can describe the eight competencies fitting all threecompetency clusters. The actual PISA framework does this in detail (meaning eightcompetencies as they play out at the reproduction cluster, at the connections cluster
and at the reflection cluster.
Finally, we want to make the observation that the competencies needed for ML are
actually the competencies needed for mathematics as it should be taught. Were that thecase (with curricula following the suggestions made by Schoenfeld and Hughes-Hallett
and extrapolating from experiences in the Netherlands and other countries), the gap
between mathematics and mathematical literacy would be much smaller than somepeople suggest it is at present (Steen, 2001). It must be noted, however, that in mostcountries this gap is quite large and the need to start thinking and working toward anunderstanding of what makes up ML is barely recognized. As Neubrand et al. (2001)
noted in talking about the situation in Germany: ‘In actual practice of German
mathematics education, there is no correspondence between the teaching ofmathematics as a discipline and practical applications within a context’ (freetranslation by author).
AN EXAMPLE: TIDES
Natural phenomena should play a vital role in mathematics for ML. For a country likethe Netherlands, with 40 percent of its area below sea level, the tides are very important.
The following protocol is taken from a classroom of 16-year-olds (De Lange, 2000):

Fig. 4: Tides

Teacher:Student A:
Teacher:
Student A:
Teacher:
Student A:
Teacher:
Student A:
Teacher:
Student B:
Teacher:
Student B:
Teacher:
Student C:
Teacher:
Student C:
Teacher:
Student D:

Teacher:
Student E:

Teacher:
Student F:
Teacher:
Student G:

Let’s look at the mean tidal graph of Flushing. What are the essentials?
High water is 198 cm, and low is -182 cm.
And? [pause]
So it takes longer to go down than up.
What do you mean?
Going down takes 6 hours 29 minutes, up only 5 hours 56 minutes.
OK. And how about the period?
6 hours 29 and 5 hours 56 makes 12 hours 25 minutes.
Now, can we find a simple mathematical model?
[pause] Maybe 2 sin x.
What is the period, then?
2 pi, that means around 6.28 hours [pause] 6 hours 18 minutes [pause] oh, I
see that it must be 2x, or, no, x/2.
So?
2 sin (x/2) will do.
Explain.
Well, that’s simple: the maximum is 2 meters, the low is -2 meters, and theperiod is around 12 hours 33 minutes or so. That’s pretty close, isn’t it?
[to the class] Isn’t it?
No, I don’t agree. I think the model needs to show exactly how high thewater can rise. I propose 190 sin (x/2) + 8. In that case, the maximum isexactly 198 cm, and the minimum exactly -182 cm. Not bad.
Any comments, anyone? [some whispering, some discussion]
I think it is more important to be precise about the period. 12 hours 33
minutes is too far off to make predictions in the future about when it will behigh water. We should be more precise. I think 190 sin [(pi/6.2)x] is muchbetter.
What’s the period in that case?
12.4 hours, or 12 hours 24 minutes, only 1 minute off.
Perfect. What model do we prefer? [discussion]
190 sin [(pi/6.2)x] + 8.

The discussion continued with ‘What happens if we go to a different city that has
smaller amplitudes and where high tides come two hours later?’, ‘How does this affectthe formula?’, ‘Why is the rate of change so important?’
Why do we consider this a good example of mathematics for ML? Given thecommunity in which this problem is part of the curriculum, the relevance for society isimmediately clear – and the relevance is rising with global temperatures. The relevancealso becomes clear at a different level, however. The mathematical method of trial anderror illustrated here not only is interesting by itself, but the combination of the methodwith the most relevant variables also is interesting: in one problem setting we areinterested in the exact time of high water; in another we are interested in the exact
height of the water at high tide. Intelligent citizens need insight into the possibilities
and limitations of models. This problem worked very well for these students, age
sixteen, and the fact that the ‘real’ model used forty different sine functions did not
really make that much difference with respect to students’ perceptions.
One important observation has to be made: one should not confuse ML with the studyof the discipline of mathematics. ML is about the functionality of the discipline as thestudents encounter their discipline at school. But the aspects of abstraction,formalization and structure of mathematics are often overlooked at school as well, andthis aspect is well worth another study. But the example also shows that some real goodtraditional mathematics is really needed to become literate in this example, for thiscertain community of learners.
PROBLEMS AND PROMISESOne of the main aspects of PISA that seems to draw fire a lot is the ‘horse-race aspect’.
PISA is seen as a competition with winners and losers and indeed, the report by theOECD is quite supportive of this judgment. On the first pages it is talking about
winners and losers. And by doing this it gives authors like Clarke (2003) a goodfoundation for their critique that with PISA and TIMSS there is a ‘current
preoccupation with competition’.
Clarke is right of course if we see that the TIMSS-report says: ‘Singapore was the
top-performing country’ (on page 3, in 2003) and the PISA report is even faster:‘Finland is the top performing country’, on page 1 (!).
This horserace might be of some interest to the popular press (Der Spiegel: ‘The
Winner is Finland’) but it overwhelms the real questions, about the instrument, forinstance. Keitel and Kilpatrick (1999) observe, quite correctly, that ‘questions ofcontent have usually been seen as secondary’. This is in line with Bonnet’s (2002)conclusion that ‘one would have wished for more space to be devoted to trying tounderstand what happens in the classrooms in terms of teaching and learning.’ Given
my earlier published comments like: ‘concerns about content validity resulted in somecountries in national options’, and: ‘the content is a point of concern that gets relatively
little attention’, I do agree with the earlier quoted authors (De Lange, in press).
But not only the horserace, and the lacking interest in content and instrument aspects isworrying a lot of observers. There are also validity aspects that need further discussion.
A main issue seems to be the choice to measure ‘mathematical literacy’ instead of‘curricular mathematics’ as in TIMSS. One certainly can join OECD in there effort tomeasure mathematical literacy, as defined in the frameworks of OECD. It seems aworthwhile effort to try not only to define the functionality of mathematics for 15 year
old students, but also to design an instrument that tries to operationalize this definition

in tasks. But how do we know for sure that these tasks will do the job? Can we reallymeasure mathematical literacy with multiple choice and open questions, with anaverage answer time of less than 2 minutes? Is it true that we need other instruments aswell, like extended tasks (like in PISA Problem Solving). Like group work, like IT use
in solving the literacy problems?
And talking about the instrument: how does OECD decide what the cut point will be to
be called literate? One can easily find arguments that one test for students of so many
countries (in 2006 about 70) cannot suffice the needs of all countries. For instance, in asecondary analysis of the PISA 2003 mathematics result, the researchers suggest thatfor the Netherlands one can only be called literate if above the level 4 from OECD.
OECD has her cut point above level 1.
A point that is often ignored about the horserace that indeed we see Finland reach thefinish line first, but do we know the ‘time’ or whether or not it is a world record? Inother words, the scores in PISA are relative scores: the average over all countries is
500 and the standard deviation is 100. So, although the definition of mathematical
literacy is quite elaborated in a whole variety of competencies, there is no criterionreferenced testing in PISA. Everything is norm referenced and we have no clue
whether or not we do really well even if we are a top-performer as a country. To get abetter view on the competencies of students in a country, we need secondary analysis
of scores and insight in actual student’s work. OECD encourages the use of their data
for this purpose but even if secondary analysis takes place, it never get the media
attention it deserves, and does not change in any respect the horserace.
A well acknowledged problem of any international comparative test is the fact that
education is culturally embedded. Clarke’s remark: ‘to assess the extend to which
Ethiopian students possess the mathematical skills required for effective participationin American society seems both futile and uninformative’ is often embraced but rather
futile itself. The mathematical competencies as identified in the Assessment
Framework try go beyond the cultural aspects: to think somewhat logically, to be
critical, to reason mathematically, to be able to use mathematics are competencies thatare useful in any situation or culture. In our opinion the cultural aspect is less about the‘content’ and much more about the ‘cultural context’ of education in a country.
The following graph, showing socio/economic/geographical clustering of countries
participating in TIMSS, tells a story:

Fig. 5: Mathematical achievement in TIMSS
Authors also get quite excited about the role of contexts in large-scale assessments.
There are many good reasons to do so, as we still fail to understand quite often theactual role the context plays in a certain problem. As Feijs and De Lange (2004)
concluded: ‘the choices of context in relation to item construction and validation is avery complex one’. And I would like to add: we cannot say anything firm about therelationship ‘context familiarity’ to ‘success rate’.
Officially the PISA study is about measuring or evaluating educational systems. But
this is not always simple. Just take the case of Belgium and the Netherlands that doequally well in the horserace but have very different educational systems. The ‘one
number tells all principle’ can backfire in bad ways, if we leave it at that one number.
This adds to the undesirability of the horserace.
A point of concern is also the choice of what is in the report and what is left out. It isinteresting to know that PISA is overseen by a board that consists of officials from thedifferent countries that represent their departments of education. This isunderstandable as they are paying for the whole PISA enterprise. But at the other side
the board can influence the way the results are gathered (multiple-choice is cheap) andwhich results end up in a prominent place in the reports. As the political scientist Kettle
once observed, educational tests are not really about measurement but about politicalcommunication.
The Dutch government once asked us to give a reason or two why Singapore
outperforms the Netherlands (again: the horserace). One of the reasons might be foundif we analyze a graph, that we constructed ourselves, but that was based on available
data from TIMSS. Why is this graph not in the report? Who makes the choices?
16
SIN% of time spent on math

15
EST

14

BEL
HK

13

KOR
TP

12
HON

11

JAP
NL

10
MAL

9
8
7
480

500

520

540

560

score on TIMSS

580

600

620
country

Fig. 6: TIMSS score versus time spent on math
Nevertheless, in our opinion PISA can become an important instrument for measuring
mathematical literacy. If it does so at present remains to be proven, but it does somevaluable things. One of the most important is putting the volume of ML at the heart ofthe discussion.
And PISA departs from the usual curricular approach and tries to find out how well our
young students are equipped for modern society. It has an innovative and ambitious
framework for assessment with the main components: Content, Context andCompetencies. It has a nice collection of items that make up the instruments, incomparison with items from previous large scale assessments. It lends itself toextensions that will make the validity better, and make the information more reliable
and relevant. It may influence, in a positive way, the thinking about the value ofmathematics education in a substantial part of the world, which might in return lead tobetter communication about the math education problems in the world. It might
support the very necessary communication between psychometricians at one side, andmath educators at the other. If secondary analyses are carried out the relevance forindividual countries will be enhanced considerably.
A FINAL WORD
A final word of warning should be placed here, because heated discussions can arise
from being uninformed. OECD/PISA has been clear in pointing out that PISA
measures mathematical literacy, which is not the same as curricular mathematics.
However, in the popular media this distinction almost never is reported. Therefore,sometimes OECD has been found ‘guilty’ for imposing a new curriculum formathematics for its member states, and the guests.
PISA reports how students perform on tasks that are intended to measure mathematical
literacy, and it is clear therefore that it does not tell how well students ‘master’ theirschool curricula. TIMSS does that better.
It is also clear that many countries take the outcomes of PISA seriously in the sensethat they embrace the idea that the output of an educational process should include a
certain amount of ‘functionality’. But it is up to the countries to decide how important
this aspect is. And from that policy standpoint will follow how much attention will begiven to the functionality in the school curriculum. One country that has taken thesesteps is Germany: this country made great efforts in introducing more context oriented
problems involving good mathematics. The score from 2000 to 2003 has improved, so
it will be interesting what Germany will do in 2006 and 2009. Other countries will dosimilar actions. But we should not forget that the more formal and abstract
mathematics, showing the structure of the discipline is not what PISA is all about –although some ‘inner-mathematical’ problems are included.
Many countries consider mathematics education having three primary goals:
preparing students for society;preparing students for future schooling and work;
showing the students the beauty of the discipline.
PISA addresses primarily the first two goals. It would be an additional challenge to see
if one can design an international study that addresses the third goal. It seems that theInternational Mathematics Union could take an initiative in this direction.
References
Boertien, H., & De Lange, J. (1994). The national option of TIMSS in the Netherlands.
Enschede, the Netherlands: University of Twente.
Bonnet, G. (2002). Reflections in a Critical Eye: On the Pitfalls of International Assessment.
Assessment in Education: Principles, Policy and Practice, 9 (3), 387-399.
Clarke, D. (2003). International Comparative Research in Mathematics Education. In A.J.
Bishop, M.A. Clements, C. Keitel, J. Kilpatrick, & F.K.S. Leung (Eds.), Second
International Handbook of Mathematics Education, Part one (pp. 143-184).
Dordrecht/Boston/London: Kluwer Academic Publishers.
Cockroft, W. H. (1982). Mathematics Counts. Report of the Committee of Inquiry into the
Teaching of Mathematics in Schools. Londen: Her Majesty’s Stationery Office.
De Lange, J. (1992). Higher Order (Un-)Teaching. In Developments in School Mathematics
Education around the World, edited by I. Wirszup and R. Streit, vol. 3, 49-72. Reston, VA:
National Council of Teachers of Mathematics (NCTM).

De Lange, J. (1995). Assessment: No Change Without Problems. In T.A. Romberg (Ed.)
Reform in School Mathematics and Authentic Assessment (pp. 87-172). New York, NY:
State University of New York Press.
De Lange, J. (1999). Framework for Assessment in Mathematics. Madison, WI: National
Center for Improving Student Learning and Achievement in Mathematics and Science(NCISLA).
De Lange, J. (2000). The Tides They are A-Changing. UMAP-Journal 21(1), 15-36.
De Lange, J. (in press). Large-Scale Assessment and Mathematics Education. In D. Grouws(Ed.), Handbook of Research on Mathematics Teaching and Learning.
Devlin, K. J. (1994). Mathematics, the science of patterns: The search for order in life, mind,
and the universe. New York: Scientific American Library.
Ellis, Jr., W. (2001). Numerical Common Sense for All. In L.A. Steen (Ed.), Mathematics and
Democracy: The Case for Quantitative Literacy (pp. 61-65). Princeton, NJ: National
Council on Education and the Disciplines.
Ewell, P.T. (2001). Numeracy, Mathematics, and General Education. In L.A. Steen (Ed.),
Mathematics and Democracy: The Case for Quantitative Literacy (pp. 37-48). Princeton,
NJ: National Council on Education and the Disciplines.
Feijs, E. & De Lange, J. (2004). The Design of Open-Open Assessment tasks. In T. Romberg(Ed.), Standards-based mathematics assessment in middle school: rethinking classroom
practice (pp. 122-136). New York, NY: Teachers College Press.
Freudenthal, H. (1973). Mathematics as an Educational Task. Dordrecht: Reidel.
Grünbaum, B. (1985). Geometry Strikes Again. Mathematics Magazine 58(1), 12-18.
Hughes-Hallett, D. (2001). Achieving Numeracy: The Challenge of Implementation. In L. A.
Steen (Ed.), Mathematics and Democracy: The Case for Quantitative Literacy (pp. 93-98).
Princeton, NJ: National Council on Education and the Disciplines.
International Life Skills Survey (ILSS). (2000). International Life Skills Survey. Ottawa,
Canada: Statistics Canada, Policy Research Initiative.
Keitel, C., & Kilpatrick, J. (1999). The Rationality and Irrationality of International
Comparative Studies. Chapter 16. In G. Kaiser, E. Luna & I. Huntley (Eds.), International
Comparisons in Mathematics Education (pp. 241-256). London: Falmer Press.
Kennedy, D. (2001). The Emperor’s Vanishing Clothes. In L. A. Steen (Ed.), Mathematics
and Democracy: The Case for Quantitative Literacy (pp. 55-60). Princeton, NJ: National
Council on Education and the Disciplines.
Ministerio de Educación y Ciencia. (1990). Ley de Ordenación General del Sistema
Educativo (LOGSE). Madrid, Spain: Ministerio de Educación y Ciencia.
Mathematical Sciences Education Board (MSEB). (1993). Measuring What Counts: A
Conceptual Guide for Mathematical Assessment. Washington, DC: National Academy
Press.
National Center for Education Statistics. (1993). National Adult Literacy Survey. Washington
D.C.: National Center for Education Statistics (NCES).

National Council of Teachers of Mathematics (NCTM). (1989). Curriculum and Evaluation
Standards for School Mathematics. Reston, VA: National Council of Teachers ofMathematics (NCTM).
National Council of Teachers of Mathematics (NCTM). (2000). Principles and Standards for
School Mathematics. Reston, VA: National Council of Teachers of Mathematics (NCTM).
Neubrand, N. et al. (2001). Grundlager der Ergänzung des Internationalen
PISA-Mathematik-Tests in der Deutschen Zusatzerhebung. ZDM 33(2), 45-59.
Niss, M. (1999). Kompetencer og Uddannelsesbeskrivelse
Subject-Description]. Uddanneise 9, 21-29.

[Competencies

andOrganization for Economic Cooperation and Development (OECD). (1999). Measuring
Student Knowledge and Skills. A New Framework for Assessment. Paris: OECD.
Organization for Economic Cooperation and Development (OECD). (2002). Framework forMathematics Assessment. Paris: OECD.
Schneider, C.G. (2001). Setting Greater Expectations for Quantitative Learning. In L.A.
Steen (Ed.), Mathematics and Democracy: The Case for Quantitative Literacy (pp.
99-106). Princeton, NJ: National Council on Education and the Disciplines.
Schoenfeld, A.H. (2001). Reflections on an Impoverished Education. In L.A. Steen (Ed.),
Mathematics and Democracy: The Case for Quantitative Literacy (pp. 49-54). Princeton,
NJ: National Council on Education and the Disciplines.
Steen, L.A. (1990). On the Shoulders of Giants: New Approaches to Numeracy. Washington,
DC: National Academy Press.
Steen, L.A. (2001). Mathematics and Democracy: The Case for Quantitative Literacy.
Princeton, NJ: National Council on Education and the Disciplines.
Treffers, A. (1991). Meeting Innumeracy at Primary School. Educational Studies inMathematics 22, 333-352.
Tufte, E.R. (1983). The Visual Display of Quantitative Information. Cheshire, CT: Graphic
Press.
Tufte, E.R. (1990). Envisioning Information. Cheshire, CT: Graphic Press.
Tufte, E.R. (1997). Visual Explanations – Images and Quantities, Evidence and Narrative.
Cheshire, CT: Graphic Press.