PROJECT MANAGEMENT BY MULTIMOORA AS AN INSTRUMENT FOR TRANSITION ECONOMIES / PROJEKTŲ VADYBA SU MULTIMOORA KAIP PRIEMONĖ PEREINAMOJO LAIKOTARPIO ŪKIAMS

<jats:p>The countries of Central and Eastern Europe moved from a previously centrally planned economy to a modern transition economy with strong market aspects. This paper proposes project management as an answer to this transition. Traditional Cost‐Benefit analysis does not respond to this purpose. Indeed Cost‐Benefit analysis is only interested in one specific project and not in a competition between projects. In addition all goals (objectives) have to be translated into money terms, leading sometimes to immoral consequences. On the contrary Multi‐Objective Optimization takes care of different objectives, whereas the objectives keep their own units. However different methods exist for the application of Multi‐Objective Optimization. The authors tested them after their robustness resulting in seven necessary conditions. MOORA (Multi‐Objective Optimization by Ratio analysis) and MULTIMOORA (MOORA plus Full Multiplicative Form), assisted by Ameliorated Nominal Group and Delphi Techniques, satisfy the seven conditions, although in a theoretical way. A simulation exercise illustrates the use of these methods, ideals to be strived for as much as possible. Santrauka Centrinės ir rytų Europos šalys perėjo iš anksčiau centralizuotai planuojamos ūkinės sistemos į šiuolaikinę pereinamąją ūkinę sistemą, kuriai būdingi ryškūs rinkos požymiai. Šiame straipsnyje siūloma projektų vadyba kaip atsakas į perėjimą. Įprastinė kainos ir naudos analizė tam tikslui yra netinkama. Be viso to, kainos ir naudos analizėje neatsižvelgiama į kiekvieną atskirai paimtą projektą bei į konkurenciją tarp tų projektų. Visi tikslai turi būti pakeisti piniginėmis vertėmis. Del to kartais kyla nepageidaujamų pasėkmių. Priešingai tam daugiatikslė optimizacija atsižvelgia į skirtingus tikslus, išsaugant tikslams būdingus mato vienetus. Yra daug įvairių daugiatikslės optimizacijos metodų. Autoriai patikrino jų stipriasias savybęs pagal septynias būtinasias sąlygas. MOORA (daugiatikslė optimizacija santykių dydžių analizės pagrindu) ir MULTIMOORA (MOORA plius pilnoji sandaugos forma), apimanti patobulintą normaliųjų grupių ir Delphi būdus, geriausiai atitinka septynias būtinąsiąs sąlygas taip pat ir teoriniu lygmeniu. Pavyzdžio modelis iliustruoja šiu metodu taikymą, idealai buvo pasiekti tiek, kiek galima.</jats:p>

section_content: In a broader sense China could be considered as a Transition Economy, though the narrower definition of Transition Economy is mostly used. "The transition from plan to market in postcommunist countries is an economic transformation of remarkable scale. Starting around 1990, countries of the former Soviet Union and of Central and Eastern Europe removed central planning, liberalized prices and foreign trade; and introduced modern institutions of taxation, banking, customs and independent central banking. Since that time, the typical transition country has privatized the majority of its industrial enterprises" (Guriev, Zhuravskaya 2009: 143). Vorapajev already made a Russian case study on project management development in that country (1998). We think rather of the transition economies in Central and Eastern Europe. 

In this paper project management, which is generally used in a market economy, is proposed as an answer to this transition. Project Management assumes "that the project to be analysed will constitute a new economic activity … in practice, however, many projects will only modify an existing economic activity" (UN Industrial Development Organization, 5). In addition different competing projects are considered and a final choice is made by Multiple Objective Optimization. 

Project Management is subject of an evolution concerning the objectives to strive after. If before the stress was put on market analysis, net present value, internal rate of return and other micro-economic targets, macro-economic objectives receive more and more attention such as employment, value added and the influence on the balance of payments. Attention for social well being goes even a step further with for instance environment and pollution. Employment is a human right, sometimes even written down in national constitutions, such as in the Belgian Constitution under article 23. The grief experienced by a widow when she lost her husband in a car accident is not compensated by insurance payments. One has rather to put as a target the minimization of car accidents. 

On these moral grounds cost-benefit, which translates costs as well as benefits into money terms, can not deliver a solution. 

Contrary to Cost-Benefit, Cost-Effectiveness can take different projects into account simultaneously. The analysis, however, is bi-objective: costs expressed in a common monetary unit on the one side and a single effectiveness-indicator on the other. Cost-Effectiveness is much used in defense: a weapon system could balance costs against the rate of kill, expressed in one or another military indicator (Brauers 1976: 67-126). As initially optimality was absent in cost-effectiveness, several addenda were proposed. 

First, Lange launched his Economic Principle. The Economic Principle of Lange runs like this: either costs are kept constant with maximization of an objective (Effectiveness), or effectiveness is kept constant with minimization of costs (Efficiency) (1968: 66-70). From linear programming it is known that for this dual the solution is identical, only an assumption for nonlinear systems. At that moment, the question remains if an optimal solution is found. 

Second, Fractional Programming forms a substitute for the dual problem 1 : max. E / C = max. Effectiveness / min. Costs. 

In Fractional Programming not all objectives are included. In econometrics a single Disturbance Term includes all the remaining objectives (Kennedy 1998: 2). Even if the disturbance term is taken as an objective only three objectives are taken into consideration. In most multiple objectives studies more than three objectives are considered. Indeed next example includes more than three objectives.
section_name: project management in a transition economy

section_content: Let us take the example of the city of Wuhan, Hubei Province, China. The city of Wuhan has a very important strategic position being located more or less on an equal distance from Shanghai in the East, Hong Kong and Guangzhou in the South, Chengdu in the West and Beijing in the North. 

In 1994 an international conference was held in Wuhan under the title: "High Technology and its Developing Strategies". A paper was delivered as: "Scenarios for Updating or Replacement of Old Industry" (Brauers 1994 ). In the same year in the WISCO steel plant a factory was in operation bought from Cockerill, Belgium (now Mittal) which was considered in Belgium as outdated. Nowadays the situation is entirely reversed with a high growth rate and more emphasis on new projects, even with competition between different new projects. 

Suppose an interest for a seaport in Wuhan. The fact that sea ships could come far inside China is an important advantage. Indeed Wuhan is located a proximally 1,000 km inland. The bottlenecks are rather of a technical kind. If a city like Wuhan would like to become an inland seaport different existing too low bridges have to be changed and the course of the Chang Jiang (Yangtze River) has to be corrected. The traditional used method is a Cost-Benefit Analysis with the following disadvantages. 

1) In traditional cost-benefit analysis, only one single project is examined without looking after other projects or other alternative uses;

2) Cost-Benefit Analysis takes a monetary unit as the common unit of measurement for benefits and costs. Indeed, even benefits are expressed in the chosen monetary unit, either in a direct or in an indirect way. In this way, cost-benefit presents a materialistic approach, whereby for instance unemployment and health care are degraded to monetary items, which is even immoral with a human life translated in the results of an insurance contract. Nevertheless cost-benefit is used in many transport models. People are more easily solution-minded than objective-oriented. Cost-benefit analysis is a product of this way of thinking. Cost-benefit studies will have fewer and fewer chances today than before. 

Multi-Objective Optimization will take care of the disadvantages of Cost-Benefit in particular for a seaport. 

1) All possible locations of seaports will be considered:

-A first alternative consists of the installation at a riverside port, inland and on the river itself, capable of receiving huge ships. The possibility to bring the huge ships so far inland is an important advantage of this project, reflected in the willingness of the ship owners to pay high demurrage and local taxes for this solution. The Port of Antwerp, the second of Europe, is located 80 km inland with partly open docks and partly locks. 

On the contrary, Rotterdam, the first port of Europe is located immediately near the sea and in open docks 2. -A second alternative possesses the same advantages as the first belonging also to a riverside port but installed behind locks. This project also means fewer problems with low and high tide, but investments costs are higher given the necessity to foresee locks and docks. -A third alternative is located as a seaport immediately near the sea but behind locks, which means fewer tidal problems, but again with huge investment costs. -A fourth alternative consists of a terminal also immediately near the sea but in open docks i. e. without locks. This alternative means fast delivery of the goods but with a severe problem of salinity, caused by the open dock system at the seaside. -A fifth alternative would mean a container terminal on a dam or on an artificial island in sea with transshipping from huge to smaller ships and fast delivery of the goods. Investment costs however are extremely high translated into high depreciation costs for the dam or the island. 2) Even broader, other investment opportunities in the country have also to be taken into consideration (Adler 1967). 

It could be that the government instead of financing a new port for instance likes to spend money on the economy and on the infrastructure of the Xinjiang Territory given the unrest in that part of the country. 

3) The objectives can keep their own units In order to define an objective better we have to focus on the notion of attribute. Keeney and Raiffa (1993: 32-38) present the example of the objective "reduce sulfur dioxide emissions" to be measured by the attribute "tons of sulfur dioxide emitted per year". An attribute should always be measurable. Simultaneously we aim to satisfy multiple objectives, whereas several alternative solutions or projects are possible, characterized by several attributes. An alternative should be quantitatively well defined. An attribute is a common characteristic of each alternative such as its economic, social, cultural or ecological significance, whereas an objective consists in the optimization (maximization or minimization) of an attribute. The term "criteria", in the meaning of desirability, is a bit weaker than objectives. 

Economics of Welfare (the term was invented by professor Pigou 1920) comprises microand macroeconomics. Microeconomics would include attributes such as: yearly capacity to be reached, net present value (NPV), internal rate of return (IRR) and payback period. Macroeconomics would include increase in Gross Domestic Product (GDP), surplus in the current account of the balance of payments, direct and indirect employment increase and ENPV. Indirect employment is measured by input-output techniques. ENPV means Economic Net Present Value, i. e. discounted revenues before national taxes, minus discounted investments, exclusive of subsidies. ENPV is different from GDP, but represents in macro-economics the counterpart of NPV, also with deduction of investments. 

Sustainable development would include: no overproduction due to capacity installed in all the national ports together, banning of all kind of pollutants and amelioration of the quality of life. 

Satisfaction of all stakeholders is still another series of objectives. Stakeholders mean everybody interested in a certain issue, here namely seaport planning. Due to consumer sovereignty and the economic law of decreasing marginal utility, consumer surplus, level of salaries, leisure time and again employment at the local and national level have to be taken into consideration. In addition, conflicts may arise between local and national authorities and port authorities and between the protagonists for a more commercial port against those for a rather more industrial one. Conflicts between all these points of view have to be avoided. Anyway, a government may consider the seaport as an industrial promotion zone. For a port industrialization means an element of stability in its port traffic. For the industrialist it means better and effective communication with his clients. The point of view of an investor who has to choose between a project of industrial or commercial development in a port and a project of industrial or commercial development elsewhere presents still another point of view. Consequently, port planning is multi-objective in many fields. 

Some attributes like NPV, ENPV, GDP, balance of payments surplus and consumer surplus are expressed in money terms, like dollars or Euros. However, a Euro in consumer surplus cannot be compensated for instance with a GDP-Euro. In addition, IRR is expressed in a percentage, the payback period in months or years, employment in number of persons per year, transport, for instance, in TEU, etc. Consequently, a serious problem of normalization is present. 

Normalization means reduction to a normal or standard state. However, the term got many interpretations but the stress is mainly put on the unification of diverting systems of measurement. As decision making is interested in measurement, normalization in technology is a main starting point, whereas scales of measurement and measurement of quality may be troublesome (for more on normalization, see : Brauers 2007).
section_name: an example of project management under multiple objectives

section_content: For the researcher in multi-objective decision support systems the choice between many methods is not very easy. Indeed numerous theories were developed since the forerunners: Condorcet (the Condorcet Paradox, against binary comparisons, 1785, LVIII), Gossen (law of decreasing marginal utility, 1853) Minkowski (Reference Point, 1896, 1911) and Pareto (Pareto Optimum and Indifference Curves analysis 1906, 1927,) and pioneers like Kendall (ordinal scales, since 1948), Roy et al. (ELECTRE, since 1966), Miller and Starr (Multiplicative Form for multiple objectives, 1969), Hwang and Yoon (TOPSIS, 1981) and Saaty (AHP, since 1988). 

We intend to assist the researcher with some guidelines for an effective choice. In order to distinguish the different multi-objective methods from each other we use the qualitative definition of robustness: the most robust one, as robust as…, simple robust, less robust than etc. with the meaning found in Webster's new Universal Unabridged Dictionary for robust: strong; stronger, strongest 3. 

The most robust multi-objective method has to satisfy the following conditions. 1) The method of multiple objectives in which all stakeholders are involved is more robust than this one with only one decision maker or different decision makers defending their own limited number of objectives. All stakeholders mean everybody interested in a certain issue (Brauers 2007: 454-455). Consequently, the method of multiple objectives has to take into consideration consumer sovereignty. The method taking into consideration consumer sovereignty is more robust than this one which does not respect consumer sovereignty. Community indifference loci measure consumer sovereignty. Solutions with multiple objectives have to deliver points inside the convex zone of the highest possible community indifference locus (these solutions are defined in: Brauers 2008b: 98-103);

2) The method of multiple objectives in which all non-correlated objectives are considered is more robust than this one considering only a limited number of objectives (Brauers, Ginevičius 2009: 125-126) ;

3) The method of multiple objectives in which all interrelations between objectives and alternatives are taken into consideration at the same time is more robust than this one with interrelations only examined two by two (for the proof of this statement, see : Brauers 2004: 118-122) ;

4) The method of multiple objectives which is non-subjective is more robust than this one which uses subjective estimations for the choice and importance of the objectives and for normalization. 

4. 1) For the choice of the objectives A complete set of representative and robust objectives is found after Brain Storming and Ameliorated Nominal Group Technique Sessions with all the stakeholders concerned or with their representative experts (see Appendix A).
section_name: The seven conditions of robustness in multi-objective methods

section_content: The method of multiple objectives which does not need external normalization is more robust than this one which needs a subjective external normalization (Brauers 2007). Consequently, the method of multiple objectives which uses non-subjective dimensionless measures without normalization is more robust than this one which uses for normalization subjective weights (weights were already introduced by Churchman, Ackoff in 1954 and Churchman et al. 1957) or subjective non-additive scores like in the traditional reference point theory (Brauers 2004: 158-159) 4. 4 The additive method with weights starts from the following formula:

(1) U j = overall utility of alternative j with j = 1, 2,. . ., m, m the number of alternatives, w i = weight of attribute i indicates as well as normalization as the level of importance of an objective

. ., n; n the number of attributes and objectives, x ij = response of alternative j on attribute i. Reference Point Theory is not linear, whereas non-additive scores replace the weights. The non-additive scores take care of normalization:

4. 3) For giving importance to an objective With weights and scores importance of objectives is mixed with normalization. Indeed weights and scores are mixtures of normalization of different units and of importance coefficients. On the contrary Delphi can determine the importance of objectives separately from normalization. In addition, as all stakeholders concerned are involved, the Delphi method is non-subjective (see Appendix B). 

5) The method of multiple objectives based on cardinal numbers is more robust than this one based on ordinal numbers: "an ordinal number is one that indicates order or position in a series, like first, second, etc. " (Kendall and Gibbons 1990: 1). Robustness of cardinal numbers is based first on the saying of Arrow (1974: 256) : "Obviously, a cardinal utility implies an ordinal preference but not vice versa" and second on the fact that the four essential operations of arithmetic: adding, subtracting, multiplication and division are only reserved for cardinal numbers (see also : Brauers, Ginevičius 2009: 137-138). 

6) The method of multiple objectives which uses the last recent available data as a base is more robust than this one based on earlier data (Brauers, Ginevičius 2009: 133, 2 th ). 

7) Once the previous six conditions fulfilled the use of two different methods of multi-objective optimization is more robust than the use of a single method; the use of three methods is more robust than the use of two, etc. 

Consequently we have to find a method which satisfies all conditions, inclusive the seventh condition. This is the case with MOORA (Multi-Objective Optimization by Ratio Analysis) composed of two methods and eventually assisted with the Ameliorated Nominal Group Technique and with Delphi. Up till now no other theory is known including three or more methods.
section_name: 4.2) For Normalization

section_name: multi-objective optimization by ratio analysis (mOORa)

section_content: The method starts with a matrix of responses of different alternatives on different objectives:

with: x ij as the response of alternative j on objective i; i = 1, 2, …, n as the objectives; j = 1, 2, …, m as the alternatives. 

MOORA goes for a ratio system in which each response of an alternative on an objective is compared to a denominator, which is representative for all alternatives concerning that objective. For this denominator the square root of the sum of squares of each alternative per objective is chosen (Van Delft and Nijkamp 1977) 5 :

s i = the score of objective i indicates as well as normalization as the level of importance of an objective (i = 1, 2,. . . n, n the number of objectives) normalized

The distance is measured between x ij * and the corresponding coordinate of a chosen Reference Point r i. 5 Brauers, Zavadskas, Peldschus and Turskis (2008: 188-190) prove that the square root of the sum of squares of each alternative per objective is the best choice for the denominator in the ratio system. 

with: x ij = response of alternative j on objective i; j = 1, 2,. . ., m; m the number of alternatives; i = 1, 2, …n; n the number of objectives; x ij * = a dimensionless number representing the normalized response of alternative j on objective i. These dimensionless responses of the alternatives on the objectives belong to the interval [0; 1]. 

Dimensionless Numbers, having no specific unit of measurement, are obtained for instance by deduction, multiplication or division. The normalized responses of the alternatives on the objectives belong to the interval [0; 1]. However, sometimes the interval could be [-1; 1]. Indeed, for instance in the case of productivity growth some sectors, regions or countries may show a decrease instead of an increase in productivity i. e. a negative dimensionless number 6. 

For optimization these responses are added in case of maximization and subtracted in case of minimization:

with: i = 1, 2, …, g as the objectives to be maximized; i = g + 1, g + 2,…, n as the objectives to be minimized; y j * = the normalized assessment of alternative j with respect to all objectives. 

An ordinal ranking of the y j shows the final preference. 

For the second part of MOORA the Reference Point Theory is chosen with the Min-Max Metric of Tchebycheff as given by the following formula (Karlin and Studden 1966: 280) 7 :

This reference point theory starts from the already normalized ratios as defined in the MOORA method, namely formula (4).
section_name: The two parts of mOORa

section_content: It may look that one objective cannot be much more important than another one as all their ratios are smaller than one (see formula 4). Nevertheless it may turn out to be necessary to stress that some objectives are more important than others. In order to give more importance to an objective its ratios could be multiplied with a Significance Coefficient. 

The Attribution of Sub-Objectives represents another solution. Take the example of the purchase of fighter planes (Brauers 2002). For economics the objectives, concerning the fighter 6 Instead of a normal increase in productivity growth a decrease remains possible. At that moment the interval becomes [-1, 1]. Take the example of productivity, which has to increase. Consequently, we search for a maximization of productivity e. g. in European and American countries. What if the opposite does occur? For instance, take the change from USSR to Russia. Contrary to the other European countries productivity decreased. It means that in formula (4) the numerator for Russia would have been negative with the whole ratio becoming negative. Consequently, the interval becomes: [-1, +1] instead of [0, 1]. 

7 Brauers, Zavadskas (2006: 457-462) proves that the Min-Max metric is the first choice between all the possible metrics of reference point theory. 

planes, are threefold: price, employment and balance of payments, but there is also military effectiveness. In order to give more importance to military defense, effectiveness is broken down in, for instance, the maximum speed, the power of the engines and the maximum range of the plane. Anyway, the Attribution Method is more refined than that a coefficient method could be as the attribution method succeeds in characterizing an objective better. 

For instance for employment the coefficient method is changed into two numbers characterizing the direct and the indirect side of employment separately. Anyway either the choice of Significance Coefficients or the Attribution of Sub-Objectives has to be based on a Delphi exercise in order to reach agreement with all the stakeholders (for Delphi see Appendix B).
section_name: The importance given to an objective by the attribution method in mOORa

section_content: MULTIMOORA is composed of MOORA and of the Full Multiplicative Form of Multiple Objectives and in this way as up till now no other approach is known satisfying the precious six conditions of robustness and including three or more methods, MULTIMOORA becomes the most robust system of multiple objectives optimization under condition of support from the Ameliorated Nominal Group Technique and Delphi.
section_name: mulTimOORa

section_content: MOORA (Multi-Objective Optimization by Ratio Analysis) was explained under Point 4) above.
section_name: mOORa

section_content: Besides additive utilities, a utility function may also include a multiplication of the attributes. The two dimensional u(y, z) can then be expressed as a multi-linear utility function (Keeney, Raiffa 1993: 234) :

If k yz = 0 we return to the additive form. For Keeney the additive form is rather a limiting case of the multiplicative utility function (Keeney 1973: 110). 

If k yz ≠ 0, then the utility function possesses a multiplicative part: if k yz > 0, then the mutual influence is positive, if k yz < 0, then the mutual influence has a negative effect on the utility function. This representation mixes additive and multiplicative parts. It is not related to a multiplicative utility function nor to a product form, but to a bilinear representation of the form: Σ r Σ s a rs x r y s. Indeed this representation is bilinear (and in general multi-linear) and not purely multiplicative, "since two sets of variables are involved and each appears in a linear way. . . and constant coefficients can be added to make the forms completely general" (Allen 1957: 473). 

The danger exists that the multiplicative part becomes explosive. The multiplicative part of the equation would then dominate the additive part and finally would bias the results. It could happen if the factors are larger than 1, unless the weights for the multiplicative part are extremely low. 

Considering these and the previous shortcomings, preference will be given to a method that is nonlinear, non-additive, does not use weights and does not require normalization. Will a full-multiplicative form respond to all these conditions? Econometrics is familiar with the multiplicative models like in production functions (e. g. Cobb-Douglas and Input-Output formulas) and demand functions (Teekens, Koerts 1972), but the multiplicative form for multi-objectives was introduced by Miller and Starr in 1969 (237-239). 

The following n-power form for multi-objectives is called from now on a full-multiplicative form in order to distinguish it from the mixed forms:

with: j = 1, 2,. . ., m; m the number of alternatives; i = 1, 2, …, n; n being the number of objectives; x ij = response of alternative j on objective i; U j = overall utility of alternative j. 

The overall utilities (U j ), obtained by multiplication of different units of measurement, become dimensionless. 

Stressing the importance of an objective can be done by adding an α-term or by allocating an exponent (a Significance Coefficient) on condition that this is done with unanimity or at least with a strong convergence in opinion of all the stakeholders concerned. Therefore, a Delphi exercise may help (see Appendix B). 

How is it possible to combine a minimization problem with the maximization of the other objectives? Therefore, the objectives to be minimized are denominators in the formula:, ni = the number of objectives to be minimized; with: ′ U j : the utility of alternative j with objectives to be maximized and objectives to be minimized. 

As no complete data are available for project management in a transition economy satisfying all robust conditions we shall limit us to a simulation exercise.
section_name: The full multiplicative form of multiple objectives

section_name: a simulation exercise for a transition economy

section_content: 1. Are not respected:

-Condition 1: the stakeholders interested in the issue are not consulted. 

-Condition 2: all objectives are perhaps not present. 

-Condition 4: non-subjectivity in the choice of the objectives and their importance is not guaranteed. -Condition 6: the use of recent data is not relevant.
section_name: What about limits for simulation to respect robustness?

section_content: -Condition 3: all interrelations between objectives and alternatives are taken into consideration at the same time. -Condition 4: does not need external normalization. 

-Condition 5: is based on cardinal numbers. -Condition 7: once the previous conditions 3, 4 (partly), 5 and 7 are fulfilled the use of two different methods of multi-objective optimization is more robust than the use of a single method; the use of three methods is more robust than the use of two. 

MUTIMOORA is in that case. considering NPV equal to zero at the end of the project period; 3) minimization of the payback period of NPV, expressed in years and months; 4) maximization of government income: local and direct and indirect government taxes in 10,000 $; 5) maximizing direct and indirect local and national employment; indirect employment found by local and national input-output tables in person-years; 6) maximizing the increase in Gross Domestic Product in m. $; 7) minimization of the risk on 5) and 6) in %; 8) maximization of increase in 100,000$ in the balance of Payments; 9) maximization of hard currency to be provided by foreign sources for investment, expressed in money terms (m. $). Appendices C and D give the detailed tables for MOORA and the Multiplicative Form. Neither project A, B or C is dominating, which means that a ranking has to bring the solution. Project A is the best for government income, Project C for increase in employment and project B shows an in between solution. Table 1 gives the reaction of the projects on the objectives after the MULTIMOORA approach.
section_name: Are respected:

section_content: There is a small deviation in the Reference Point part of MOORA but one may conclude for MULTIMOORA that Project A with its larger income for the government is preferred above B, an in between solution. Project C comes in the last position in spite of its favorable employment total. 

Subjectivity can still be present in the choice of the objectives and of the alternatives. Political dominance can lead to this choice, either from above in centralization or federalism or from bottom up after the substitution principle or by confederation. In absence of any form of dominance convergence of ideas could lead to non-subjectivity. However, what is meant by non-subjectivity? In physical sciences, a natural law dictates non-subjectivity without deviations. In human sciences, for instance in economics, an economic law will state the attitude of men in general but with exceptional individual deviations. Outside these human laws in the human sciences unanimity or at least a certain form of convergence in opinion between all stakeholders concerned will lead to non-subjectivity. This convergence of opinion, concerning the choice of the objectives, has to be brought not by face to face methods but rather by methods such as the Ameliorated Nominal Group Technique (see Appendix A). Alternatives have to be well defined too. If alternatives concern Projects the whole theory on Project Analysis enters into the picture. Convergence on the importance of the objectives is supported by the Delphi Method (therefore see Appendix B).
section_name: in the simulation

section_content: For a researcher in multi-objective decision support systems the choice between many methods for multi-objective optimization is not very easy. We intended to assist the researcher with some guidelines for an effective choice. In order to distinguish the different multi-objective methods from each other we use a qualitative definition of robustness comparable to: strong; stronger, strongest. 

The most robust multi-objective method has to satisfy the following conditions. 

1) The method of multiple objectives in which all stakeholders are involved is more robust than this one with only one decision maker or different decision makers defending their own limited number of objectives. All stakeholders mean everybody interested in a certain issue. Consequently, the method of multiple objectives has to take into consideration consumer sovereignty too. 

2) The method of multiple objectives in which all non-correlated objectives are considered is more robust than this one considering only a limited number of objectives. 

3) The method of multiple objectives in which all interrelations between objectives and alternatives are taken into consideration at the same time is more robust than this one with interrelations only examined two by two. 

4) The method of multiple objectives which is non-subjective is more robust than this one which uses subjective estimations for the choice and importance of the objectives and for normalization. 

For the choice of the objectives A complete set of representative and robust objectives is found after Brain Storming and Ameliorated Nominal Group Technique Sessions with all the stakeholders concerned or with their representative experts.
section_name: general conclusion

section_content: The method of multiple objectives which does not need external normalization is more robust than this one which needs a subjective external normalization. Consequently, the method of multiple objectives which uses non-subjective dimensionless measures without normalization is more robust than this one which uses subjective weights or subjective nonadditive scores like in the traditional reference point theory. 

For giving importance to an objective With weights and scores importance of objectives is mixed with normalization. On the contrary Delphi can determine the importance of objectives separately from normalization. 

5) The method of multiple objectives based on cardinal numbers is more robust than this one based on ordinal numbers. 

6) The method of multiple objectives which uses the last recent available data as a base is more robust than this one based on earlier data. 

7) Once the previous six conditions fulfilled the use of two different methods of multi-objective optimization is more robust than the use of a single method; the use of three methods is more robust than the use of two, etc. 

Multi-Objective Optimization by Ratio Analysis (MOORA), composed of two methods: ratio analysis and reference point theory starting from the previous found ratios, responds to the seven conditions. If MOORA is joined with the Full Multiplicative Form for Multiple Objectives a total of three methods is formed under the name of MULTIMOORA. In addition if MULTIMOORA is joined with the Ameliorated Nominal Group Technique and with Delphi the most robust approach exists for multi-objective optimization up to now. 

It has to be accentuated that satisfying the seven conditions signifies a theoretical goal to be reached as much as possible. Paraphrasing Steuer we can say that we are in a situation trying to optimize each objective to "the greatest extent possible" 8. A simulation exercise illustrates the use of these methods, ideals to be strived for as much as possible. 

If the Simulation Exercise for a transition economy has no practical consequences, in any case it provides a learning experience with MULTIMOORA in its triple composition.
section_name: For Normalization

section_content: The assistance by Brainstorming and by the Ameliorated Nominal Group Technique Jantsch (1967, 136) gives the following basic rules for brainstorming sessions: "1. State the problem in basic terms, with only one focal point; 2. Do not find fault with, or stop to explore, any idea; 3. Reach for any kind of idea, even if its relevance may seem remote at the time; 4. Provide the support and encouragement which are so necessary to liberate participants from inhibiting attitudes". 

In any case, an efficient reporting system is necessary to memorize the ideas (stenography or recording). In general, brainstorming is insufficient for tackling broad problems and for obtaining judgmental data. Indeed opinions can be too divergent for a consensus to be reached. 

Brainstorming must be considered too simple and too naive for tackling broad problems or for obtaining judgmental data. Brainstorming is valuable for obtaining a first approximation to find a complete set of objectives. With experts representing all stakeholders for a certain issue the results remain rather fuzzy, unless an Ameliorated Nominal Group Technique is used. 

The Nominal Group Technique, which is explained here, was ameliorated by Brauers (1987 Brauers (, 2004: 44-64) : 44-64) but the Nominal Group Technique was first elaborated by Van de Ven and Delbecq (1971).
section_name: Appendix A

section_content: The nominal group technique consists of a sequence of steps, each of which has been designed to achieve a specific purpose. 

1) The steering group or the panel leader carefully phrases as a question the problem to be researched. Much of the success of the technique hinges around a well-phrased question. 

Otherwise the exercise can easily yield a collection of truisms and obvious statements. 

A successful question is quite specific and refers to real problems. The question has to have a singular meaning and a quantitative form as much as possible. 

2) The steering group or the panel leader explains the technique to the audience. This group of participants is asked to generate and write down ideas about the problem under examination. These ideas too have to have a singular meaning and a quantitative form as much as possible. Participants do not discuss their ideas with each other at this stage. This stage lasts between five and twenty minutes. 3) Each person in round-robin fashion produces one idea from his own list and eventually gives further details. Other rounds are organized until all ideas are recorded. 4) The steering group or the panel leader will discuss with the participants the overlapping of the ideas and the final wording of the ideas. 5) The nominal voting consists of the selection of priorities, rating by each participant separately, while the outcome is the totality of the individual votes. A usual procedure consists of the choice by each participant of the n best ideas from his point of view, with the best idea receiving n points and the lowest idea the lowest point. All the points of the group are added up. A ranking is the democratic result for the whole group. The Original Nominal Group Technique can be characterized as weak robust as the participants expressed too much their personal feeling. Amelioration was proposed for that reason.
section_name: The Original Nominal Group Technique

section_content: As there was too much wishful thinking even between experts better results were obtained if the group was also questioned about the probability of occurrence of the event. In this way the experts became more critical even about their own ideas. The probability of the group is found as the median of the individual probabilities. 

Finally, the group rating (R) is multiplied with the group probability (P) in order to obtain the effectiveness rate of the event (E):

Once again, the effectiveness rates of the group are ordered by ranking. One may conclude that the Ameliorated Nominal Group Technique is more robust than the Original Nominal Group Technique. In our research it is clear that the Ameliorated Nominal Group Technique concerns the search for a complete set of representative and robust objectives and sub-objectives.
section_name: The Ameliorated Nominal Group Technique

section_content: The Delphi method is a method for obtaining and processing judgmental data. It consists of a sequenced program of interrogation (in session or by mail) interspersed with feedback of persons interested in the issue, while everything is conducted through a steering group. We advocate the most this method as it also takes care of:

-Quantitative treatment; -Expert knowledge; -Anonymity; -Convergence. Dalkey and Helmer (1963) used Delphi in its present form for the first time around 1953. The essential features of Delphi are:

1. A group of especially knowledgeable individuals (experts). 

2. Inputs with a singular meaning and quantitative as much as possible. 

3. The opinions about the inputs are evaluated with statistical indexes. 4. Feedback of the statistical indexes with request for re-estimation, also after consideration of reasons for extreme positions. 5. The sources of each input are treated anonymously. 6. Two developments: meeting and questionnaires. The organization of a meeting produces quicker results. However, the meeting has to be organized in such a way that communication between the panel members is impossible. Therefore, a central computer with desk terminals, television screen and computer controlled feedback is advisable. As an example of Delphi a music competition ended with 12 finalists (Brauers 2008a). Beside the personal preferences of the jury members, different music schools or tendencies exist. Total points and the medians were the same for the first four candidates but for the 5 th and the 6 th ranks, the laureates were reversed. However, the large diversion between the first and the third quartiles illustrated a possible frustration between the jury members for the laureates ranking 5 and 6 and the other finalists ranking 7, 8, 9, 10, 11 and 12. At that moment Delphi interferes. The voting is repeated several times. In the beginning skewness is still too large but then a new round may help. Delphi experiences a better convergence in opinion as the medians and quartiles approach more and more to one another in different rounds until convergence as much as possible is reached and automatically robustness is increased. At that moment, the ranking of the finalists in the positions 5 till 12 may be entirely reversed, but the members of the jury, like the public and the press, will be more satisfied. 

In a project of multiple objectives optimization the stakeholders or their representatives are asked to give for instance a single, double or triple importance to an objective.
section_name: Appendix B The assistance by the Delphi Technique

section_content: Simulation of project planning by MOORA
section_name: Appendix C

section_content: Edmundas K. ZaVaDSKaS is a ScD, professor, Doctor Honoris Causa of the universities of Poznan, Sankt Peterburg and Kijev, vice rector of Vilnius Gediminas Technical University (Lithuania). Member of the Lithuanian Academy of Sciences, president of the Lithuanian Operetional Research Society, president of the Alliance of Experts of Projects and Building of Lithuania. He is editor-in-chief of the journals: Journal of Civil Engineering and Management, Technological and Economic Development of Economy; editor of the International Journal of Strategic Property Management. In 1973 received the PhD degree in building structures. Assistant, senior assistant, associate professor, professor at the Department of Construction Technology and Management. In 1987 received the ScD at Moscow Civil Engineering Institute (construction technology and management). Author of 14 monographs in Lithuanian, English, German and Russian. Research interests: building technology and management, decision-making theory, automation in design, expert decision support systems.