Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design Research

Study on clarification of the design evaluation criteria
Intended for Good Design Award
Ayaki Sakuma1, Yusuke Ashizawa2, Kenta Ono1, Makoto Watanabe1
12Chiba University
Japan Institute of Design Promotion

Abstract: Recently, the meaning of design is rapidly diversifying. We design for wider range offields, which include tangible and intangible subjects, and design itself has a broader concept.
Along with the globalization of society, that of design is also developing. Activities related todesign are brisk all over the world, and international exchanges are also more active than before.
Those movements influence not only on the activity of designing but also on the side of evaluatingthem. Diversifying of designing subjects and globalization of designing activities have great
influence on evaluation of designs, therefore we are strongly requested to make the criteria andcertain viewpoints clearer.
The purpose of this study is to define the evaluation criteria for "Good Design Award". This award
is a comprehensive program for the evaluation and encouragement of design. It has a function ofpromotingdesigning

aimingindustrial development throughdesigning, thereforetheaccountabilities to the applicants are important for realization of the objective. As it is also
promoting global activities, they are advancing activity to Asia through the system cooperation
with the design organizations and the awards established in Asian countries. In order to give
explanation to the applicants and to make evaluation globally with sufficient understanding,judging committee should be given an obvious criteria and viewpoints.
Based on above needs, I observed the evaluation of 2012's award and recorded what judging
committee did. I wrote down committee members' speeches and behaviors from the video I
recorded, and analyzed that by sorting out their notable speeches or classifying their behaviors.
Through this analyze, I found two hypotheses. The first is "existence of unchangeable criteria andchangeable criteria with passage of time". There are unchangeable viewpoints which compose
base of criteria from past to present, while there are changing viewpoints composed ofaccumulating discoveries through the annual award. The second is "subtracting criteria and adding
criteria". Subtracting criteria is minimal criteria applicants must obey. Those who do not fulfill
these criteria cannot pass even though some excellent points their designs have. Adding criteria is
specific characters strongly appealed. Judging members select better applicants comparing thesecharacters.
Taking these two hypotheses into account, this study can be useful as a manual of global
evaluation and as a reference for newly appointed judging committee members, or as feedbacks toapplicants who support industrial development.
Key words: Design promotion, Design evaluation and viewpoints, Design criteria

1

3088
3075
3085
3098
3097

Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design Research

1. Introduction
The purpose of this study is to make clearer criteria and viewpoints for design evaluation. Therefore, “Good
Design Award,” which is one of the design evaluations representing Japan, was chosen, and it was analyzed byinvestigating the judging, evaluation, and selection situations. Based on the result, the viewpoint that is needed fordesign evaluation is clarified, and (new) evaluation criteria are proposed.

2. Overview
2.1 Back ground
Recently, the meaning of design is rapidly diversifying. We design for wider range of fields, which include
tangible and intangible subjects, and design itself has a broader concept.
By technical innovation, the places of craftsmanship increased and more diverse design was demanded.
Furthermore, according to the changes in people's lifestyle, the design of concepts, which do not have physicalforms, such as experience, service, and a system, came to be valued. Based on this present condition, the field
called "design" spread quickly, and it is becoming difficult to define a "good design" in general.
Along with the globalization of society, that of design is also developing. Design activities are performed withvivacity in Europe, America, and also Asia and international exchanges are also prosperous. The international
exchange of designs is standard. For example, companies invest in foreign countries, and many design workshops
in which multiple countries participate are held. In education fields, studying abroad is becoming active andprograms such as MEDes are constructed.
Those movements influence not only on the activity of designing but also on the side of evaluating them.
Diversifying of designing subjects and globalization of designing activities have great influence on evaluation ofdesigns. Design evaluation has greatly contributed development of design activities. In order to show the futureindicator of design, we are strongly requested to make the criteria and certain viewpoints clearer.

2.2 Subject of study
In order to clarify the criteria and viewpoint of design evaluation, this study inquired for “Good Design Award”
in Japan. This award is a comprehensive program for the evaluation and encouragement of design organized by
Japan Institute of Design Promotion. Choosing "good designs" from the phenomena developed variously, andshowing it to the society, it aims to lead industry, living, and the whole society to be better.
In order to achieve this purpose, clear selection criteria are needed. And, the explanation to the applicants of
the results, which acts as feedback and can achieve improvement in design, are important for realization of theobjective. Moreover, Good Design Award is advancing international activities to Asia through the cooperation
with the design award of Asian countries, and the design organization of each country. In Asian countries, thenecessity for design is increasing as economy is remarkably growing. It has a great hope also for Good DesignAward that evaluates and recommends designs. Then, in order to build a high quality life and society in eachcountry through design and contribute to advancement of the social economy of whole Asia, international
exchange and promotion activities are performed positively. In recent years, many objects not only from Japan but
from overseas have applied, and evaluations are held in foreign countries including South Korea and Taiwan. To
give explanation to the applicants and to make evaluation globally requires sufficient knowledge and deep

2

3089
3076
3086
3099
3098

Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design Researchunderstanding.
To perform the examination, judging committee should be given an obvious criteria and viewpoints. For thescreening, there are five important words summarizing the basic concept and four viewpoints to evaluate eachentry. Japan Institute of Design Promotion suggests that the criteria should change with the times. But, it alsopresents five important words as philosophy based on humane thinking.
HUMANITY --Inspiration for products and conceptual embodiments
HONESTY --Perceptiveness toward contemporary society
INNOVATION --Concepts to pioneer the future
ESTHETICS --Imagination for prosperous lifestyle culture
ETHICS --Reflecting on society and environment
These are considered as the definition of good design. With these as the central idea, they have establishedfour viewpoints while placing importance on "thinking from various points of view," in order to induce the ability
to understand the proposal during the screening.
Viewpoint for human body
Viewpoint for lifestyleViewpoint for Industry
Viewpoint for society and the environment
These are prepared not as the criteria of single focus thinking but as language to support the thinking required
for screening. However, as mentioned above, the objects, which are applied these days, are diversified andcomplicated. From the idea and viewpoint which were set up now, discussions and judgments can be made,feedback which leads to improvement in the society after giving concrete explanation are not done sufficiently.
Moreover, examination developed abroad is performed with local judging committee. For promotion ofunderstandings of judging committee who newly joined, Good design award needs viewpoints refined more thanthe present.

2.3 Study flow
As study investigation, I observed the evaluation of 2012’s award. While recording two or more judging
committee member by using video, the situation of screening was recorded asking questions occasionally. I wrote
down committee members’ speeches and behaviors from the video, and analyzed that by sorting out their notablespeeches or classifying their behaviors. Consideration is performed from this analysis and assumption was foundout based on the background of the criteria and the viewpoint of examinations. Through the past examples, theviewpoint and concept required in future design evaluation were arranged, and a possibility of making criteria
establishment is searched for.

3. Investigations and Analysis
3.1 Investigation overview
I requested cooperation with five good design award judge committee and I made them the universe. I
observed the secondary screening in front of the actual products themselves, and investigated focusing on video

3

3090
3077
3087
3100
3099

Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design Research

photography and a hearing. For three days from July 24, to July 26, 2012	
  when secondary examination is
performed, I investigated withthree support staff.

3.2 Investigation method
The recording by the video was performed about the following.
(1) Recording situations of Case-by-case Screening (Total: Three Persons)I accompanied the judging committee as well as the support staff, and recorded the situation of screening. The
one recording staff accompanied to 1 or 2 judging committees.
(2) Recording of Discussion Situation
When screening on a conference table, one fixed point camera was installed, and the situation of discussion
was recorded. In addition, when moving in order to observe objects, one recording staff accompanied.
(3) Recording of Presentation Screening Situation
One fixed point camera was installed in the examination hall, and the screening situation was recorded.

Figure1.The situation of investigation

3.3 Result of an investigation
I wrote down a judge committee word and action from the record, collected for every individual and visualized
as a conversation.

Figure2. Judge committee words and actions

4

3091
3078
3088
3101
3100

Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design Research

3.4 Analysis
Out of the arranged word and action, the keyword about evaluation was extracted and the classification
division was performed. 149 keywords were checked out of the 118 for all examination proposals. They were as
follows when they were classified. The keywords were classified into six larger classifications, and were further
classified into the category of 17 as small classifications.
(1) Evaluation viewpoint
The point, which can be the target of discussions. Deviation appeared also in the viewpoint.
Evaluation item -- Simple item. Reference was not made deeply.
Regard-- Items which should be observed, such as safety and danger, were discussed.
Usability -- Evaluation by the viewpoint of users was increased.
Expression of a proposal -- Many words about impressions or appearances were observed.
Technical matter -- It was discussed also as the argument and addition item as design.
Combination -- It was discussed about matching of evaluation viewpoints.
Cost – The cost that product takes.
Accuracy -- It was discussed most. It became a final judgment element in many proposals, and it was spoken inhigh and low evaluation.
(2) About companies
There are many discussions evaluating about the company that proposed.
Attitude of companies -- discussions are held to evaluate the attitude of companies, such as a measure and efforts.
Comprehension of a company -- They referred to the difference in what they requested and comprehensions ofcompanies.
(3) The method of proposals
They discussed the "methods of proposals" which is not restricted to items that are applied.
Intention of a proposal --They evaluated to not products but the backgrounds such as Intentional precision and fun.
Expressing powers of proposals -- It becomes of the evaluation criteria whether the proposals are fully transmitted
to the judge committee in many cases, i.e. the exhibition method, how to show data, etc.
(4) The judge committee opinion
Evaluation was judged synthetically and the intention and hope were discussed.
Evaluating points -- They discussed the number of the items and synthetic judgment instead of individual
evaluations.
Evaluation possibilities -- When proposals cannot be discussed and evaluations as design were not done,

theywere discussed in many cases with argument of technical requirements.
Judges’ Hope--They referred to development of the future of proposals as the judge committee.

(5) Change with passage of time
It is the almost same item as the novelty that is the criteria. However, many arguments related toanteroposterior relation that is not made to the same kind as one criteria, the history of companies, and the historyof design were performed.
(6) Category
The scene of discussing beforehand not the criteria but the category that judges proposals was observed.

5

3092
3079
3089
3102
3101

Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design Research

Table 1. Evaluation viewpoint
Evaluation
Item

RegardUsability

Expression ofa proposal

Technical
matterCombination

Accuracy

Awkward

Forming

Matching

Forming

Cheap

New Tec

Conception

New Tec

FlashyHow to use
Accuracy ofTechnic
Technic andEconomic

Use material

How to use
Accuracy ofTechnic
Technic andEconomic

Risk to user
Risk to
BeautyFeeling

product

Charm

Regard forConsideration

Invention

Regard for

Necessity

Expression

ComfortsafetyInteraction

Usability

troubleExtravagant

Sense of
use

Rough

Visibility

Dynamic

Improvement

SimplyAdd point

Combination
Cost

Function

Function
Reasonable
price
Fearing cost

Fine regard
Sureness

Improvement
Add point

Table 2. About companies and The method of proposals
Attitude ofcompanies

Comprehension of acompany

Intention of a proposal

Sticking to the basis

Poor understanding

Existence ofSincerity

Misunderstanding

Clear improving point

confidence

Mature consideration

Not moving

Understanding degree ofintentionｓ
Articulateness of product's
claimsExistence of document

Expression ofefforts

Accuracy of directions

Concreteness

Clarity of visions

Lack of efforts

Mistake and Correct
answer

How to catch meanings

Having question or not

Skinmping works

How to convey will

Plainness

A lot of guts
Good ways ofthinkings

Pellucidity of reasonsVIsualized situation

Satisfaction

Partial expression

ExcuseFun of intentions

Abstract

Viewpoint

Clarity of approaches
Relations of intentions andresults

Prize for efforts
Company's moving

Expressing powers of proposals

basis

Articulateness of reasons

Table 3. The judge committee opinion, Change with passage of time and Category
Evaluating
pointsNot uniqueFewevaluation
points

Evaluation
possibilities
Unevaluable inindustrial viewpoints
Receipt impossibility
as proposals

Judges’ Hope

Change with passageof timeCategory

Desiring expansion into theworld

Improved

Business

Difference with expectations

Invention not to beconnected next

Universal

6

3093
3080
3090
3103
3102

Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design ResearchMuchdeduction
Llittlededuction

Inappropriate as good
designUnevaluable bydesignNot badNot design properly

Threatened domain

Unevaluable

In face of the development to
the foreign countries

Like the initialdesignNot founddevelopment

Thinking that I want

Supersession

Not ashamed to be chosen

Orthodox

De facto standard

Old-fashioned

Standard in industry

Deteriorating

No evaluation
point
Outstandingrefinement

Impossible ofjudgment as design
Work of the craftsman

More than imaginations

evolving

Want to support

Outdatedness

Solution to
the problem

Having the history
Past results
Maturity
Novelty
New development
Commonplace

4. Consideration
4.1 User’s viewpoint and Industrial viewpoint
The direction of how the judges evaluate was roughly divided into two.
The first is the direction that discerns the object which should be recommended with a user's viewpoint. The otheris direction that discerns the object which should be recommended with the viewpoint of the whole industry.
In order to elect "good design" and to evaluate the element which the object itself has, it naturally judges with a user's
viewpoint about a fixed valuation basis. And if the Good Design Award aims at industrial development at all, in orderto evaluate "the use of the design to society", it can be said that industrial viewpoints are indispensable evaluation
criteria." Evaluation criteria", "Consideration", "Usability" which carried out the classification division are theevaluations which originated in the user. Moreover, the evaluations such as "The posture of a company," "Thecomprehension of a company to design," which evaluate the company itself and helps manufacture a better product, isan examination aiming at industrial development. The utterance of "A view to overseas" as a judge's hope and "Liking
to aid" is also evaluation based on the promotion purpose peculiar to Good Design Award that is not looked at by otherdesign awards. By having a user viewpoint, which a universal design award has, and a viewpoint of industrial
development, the evaluation serves more as promotion of the improvement in design of companies.
Therefore, it has a possibility of growing the field as feedback by leaps and bounds. It can be said that it is possible
to play a role of not mere feedback but a design consultant.

4.2 Safety and Accuracy

7

3094
3081
3091
3104
3103

Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design Research

Also in evaluation criteria, "Safety" was extracted specially and taken up because a different field in basis fromother evaluation criteria existed. Other evaluation criteria judge right and wrong as an argument, unite other argumentswith the result, and consider them in combination.
However, if it results in safety, when the item is not fulfilled, the proposal is not passed even if it has thecharacteristic which was excellent in others. Although this was not able to be checked by this investigation, it can say
the same thing even in fields, such as "Social order" and "Morality."
It is necessary to classify the minimum standard with other evaluation criteria. The "accuracy" discussed this time
most was taken up specially. After finishing many arguments, the case where it finally became a standard of judgment
was observed by many proposals. "Accuracy," which was discussed the most, was taken up specially. After finishing
many arguments, the case where it finally became a standard of judgment was observed by many proposals. Theseevaluations have played a role of an additional element, and these are in the state where it separated from the center of
the argument.
Although the evaluation criteria which take the lead in an argument, of course could also become an additionof-points element, "Accuracy" was additionally discussed by many scenes. Thus, when there was also a standard
which success or failure influences by one of the items, existence of the standard additionally added as the last
judgment has been checked.

4.3 Evaluation possibility
There are some evaluations in which, "could not be evaluated".

As the background also described, the concept ofthe diversified design may ask examination for the proposal sometimes beyond the domain. In this investigation, it had
a lot of discussions about technical requirements. Of course, technical requirements are also one of the importantelements of a design.
However, when the opinions of the proposal which should be evaluated are only technical requirements, it is a factthat it is also impossible to make a judgment as a design. Thus, argument about whether it is possible to be evaluated,or how the proposal should be categorized beforehand, or the correspondence to a complicated examination is
performed flexibly. First, discerning well the base of the evaluation criteria which judge whether it can evaluate or not
is in demand. It should put, whenever it examines the element which constitutes the base, and the element used as a
new base.

4.4 Changeable criteria with passage of time
It classified in distinction from the "Novelty" which is one of evaluation criteria.
Although the meaning as a standard newly evaluated is contained in evaluation criteria "Novelty", in many cases,
the meaning "It is a forefront" is pointed out. While changeable with passage of time also has implications called thestandard as new entry, it is the feature to also mention the standard evaluated no longer.
Even if evaluated to some extent the case where it is no longer evaluated with changes of a time even if it is theobject evaluated in the past, and now, future development may be unable to expect. It was confirmed that there was astandard which changes and goes by passing through time from the past to the future. Moreover, although it is a rarecase, also when the standard which was not evaluated in the past changes so that it may be evaluated with changes oftime, reference can be made from an award example.

8

3095
3082
3092
3105
3104

Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design Research

5. Hypotheses
5.1 Unchangeable criteria and Changeable criteria with passage of time
As consideration described, the changeable criteria with passage of time has a meaning important for design
examination complicated in recent years. Moreover, it is necessary to take in the change as structure as it says also
in the text published at the idea which the as Good Design Award of 2012 has held up, "The standard shouldchange with times." Then, existence of an unchangeable standard and the changeable criteria with passage of timeis mentioned as a base of a valuation basis. There is no less than 55 years of history in Good Design Award, theviewpoint which always serves as a criterion of judgment in the history exists, and the viewpoint from which a
sense of values changes and combines and a valuation basis also changes with changes and the social trends of thetimes exists. The changeable criteria with passage of time to the big base of an unchangeable standard isaccumulated in an annual examination. It becomes a base which change takes place to the screening criterion aftera degree next year, and stacks new discovery by repeating the new discovery and technical innovation of sense ofvalues.
There is an LED light as an example for the changeable criteria with passage of time. The LED lighting withwhich commercial production was advanced by invention of the blue LED in 1996 was evaluated quickly -- thecandidate for an award in the 2003 fiscal year will be 3 or more times than the 2002 fiscal year. However, thequality for which an LED light is asked increases now, and the product evaluated in the past and a product when
progress of technology is little are no longer evaluated. It came to carry out evaluation which gazed at the past -there is neither an initial design nor a future view -- and the future simultaneously. On the other hand, the example
evaluated when the technology by which it was not evaluated in the past, catches change of a sense of values also
exists.
"eneloop" was evaluated by first in a roll in 2007, from the communication power and the construction powerof a brand of expressing ecology thought based on the battery charger which was the existing technology. Then,evaluation as ecology thought is made natural and the product has evolved as promotion of change to a new life
style as of 2012. Thus, the past novelty may be estimated to be natural now and the past technology may beevaluated with a sense of values. However, although the influence which the standard which ages has on anannual examination is great, don't forget that the eternal standard used as a base occupies most as an evaluation
viewpoint.
5.2 Subtracting criteria and Adding criteria.
As "the safety and accuracy" of consideration described, both the standard which success or failure influencesby one of the items, and the standard additionally added as the last judgment exist. Thus, evaluating all thestandards by the same layer has inconsistency. Then, the former should be made into a subtracting criteria, thelatter should be made into an adding criteria, and a valuation method different, respectively should be developed.
A subtracting criteria is a minimum standard which an object should observe. What proposal does not fill this is
not passed even if it has the characteristic which was excellent in others however. For example, it is asked whether

9

3096
3083
3093
3106
3105

Consilence and Innovation in Design - Proceedings and Program
5th IASDR 2013 TOKYO
5th International Congress of International Association of Societies of Design Research

ethical fields, such as safety and morality, are kept or social order is not disturbed. An adding criteria is thecharacteristic against which an object appeals strongly.

6. Conclusion
Evaluation of design became more important because the places of craftsmanshipincreased in number quickly in postwar Japan.
Good design award has pursued and recorded design with changes with the ages over a long
period of time.
Complication of the design field of these days is a big turning point and reviewing of the evaluation criteria
and the viewpoint will affect future design fields. More concrete and clearer viewpoints give better explanations to
the applicants. The improvement in design of companies is promoted by returning explanations as feedbacks. As
consideration also described, the applications as not only feedbacks but design consultants can be expected, andthe further development potential exists.
Moreover, by checking the criteria, the viewpoint which the judge committee has can be identified once again.
Furthermore, it enables new judge committees to promote understandings and to activate overseas examinations
and more development of design evaluation is also expected.
Although the criteria and viewpoint of design evaluation were considered in this paper based on two
assumptions, About the usefulness or rationality, I have not come to specify a basis scientifically. Moreover, newconsideration may be found from the recorded survey content.
It is necessary to continue the consideration and examination in the future.
Acknowledgment：Cooperation of this subject graduate school was received when investigating this study.
I appreciate the direction of all that cooperated with a Japanese design promotion meeting and the staffs of Chiba
University in writing.

10

3097
3084
3094
3107
3106