# Technology Evaluation Using Modified Integrated Method of Technical Project Assessment

Stanislaw Marciniak

<figure><img src="image_1.png" /></figure>

## To cite this version:

Stanislaw Marciniak. Technology Evaluation Using Modified Integrated Method of Technical Project Assessment. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2015, Tokyo, Japan. pp.493-501, 10.1007/978-3-319-22759-7_57. hal-01431137

HAL Id: hal-01431137

https://inria.hal.science/hal-01431137

Submitted on 10 Jan 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Technology Evaluation Using Modified Integrated Method of Technical Project Assessment

Stanisław Marciniak

Warsaw University of Technology, Faculty of Production Engineering, Warsaw, Poland

iosp@wip.pw.edu.pl

**Abstract.** This paper proposes a method of evaluation of novel technologies, like ontology engineering based solutions, in conditions when a paradigm based on the balance of economy, society and environment is adopted. The evaluation applies a holistic approach. The proposed method is based on the idea of integrated efficiency evaluation of technical and organizational projects. Its key elements are the assessment measures and the management control procedures.

**Keywords.** Technology evaluation • Integrated evaluation method • Controlling

## 1 Introduction

Numerous studies research technologies with regard to: implementation, justification, evaluation, and selection [3]. In order to evaluate of technologies different tools are used: data envelopment analysis [8], strategic models [7], fuzzy models [1]. The purpose of this paper is to redesign the integrated method of evaluation, which originally addressed flexible automation [5], to enable evaluation of technical and business projects considering the ‘new economy’ paradigms, which involves perspectives of environment, society and economy, and includes the aspect of sustainable development. In order to explain the idea of the method controlling was selected as a supporting tool. Controlling proposes the permanent deviation analysis of targets, and assessment of technology along the design and application phases [9]. The evaluation is preceded by creation the measures, planning the targets and possible deviations, adopting the measuring tools, selecting the guidelines for decision processes. The conditionings for creation of the measures for technical projects evaluation are presented herein.

## 2 Integrated method of technical projects assessment

The integrated method was devised by the author in 1989 [5]. Its aim was to permit evaluation of technical and organizational projects of flexible automation. In order to conduct the process of method adaptation to the contemporary conditions, it is necessary to redefine the answer to question ‘what determines such an evaluation’. It seems that the leading elements that condition the construction of an evaluation are:
* the method of approach to the evaluation according to the binding paradigm in economy and management,

* defining the impact areas of technical and organizational project that are under evaluation, which may change due to development and globalization,

* the successive changes of criteria regarding the selection of the management methods or techniques, due to the necessity to take into account the behaviour of environment, that is characterized by fast changes and growing global character [5].

In order to meet the three determinants one needs to take into consideration the fact that evaluation of technical projects should follow a holistic approach, be comprehensive and integral. The holistic approach refers to the Aristotle's paradigm that the whole precedes its parts, so the holistic approach prevails over partial approaches. Comprehensiveness means that all possible factors of the phenomenon (project) are addressed. Integral means that various aspects and links of system elements are addressed (e.g. by the evaluation system). The definition of rules and stages of the evaluation process is as important as the features of evaluation methods and needs a lot of technical information [2].

Considering the adopted paradigm that regards the balance between economy, society and environment, the evaluation process of technical and organizational projects may comprise various number of stages. The traditional process consists of five stages characterized by sequential operations:

* identification, i.e. devising the holistic specification,

* arrangement of features according to importance (proper weights),

* overview of potential evaluation methods,

* selection of an evaluation method,

* conduction the evaluation.

The structure of this procedure has a universal character and may be used for many types of production and service processes, as well as when adopting it to modern and complex technical projects, like those exploiting the means if ontology engineering. The links between the mentioned procedures are presented in Fig.1.

<figure><img src="image_3.png" /><figcaption>Fig. 1. The structure of evaluation process of technical and organizational projects</figcaption></figure>
The rules of evaluation process include not only the procedure stages within the
process and the characteristic of the evaluation process itself (Fig.1). They also in-
clude some aspects of the evaluation process:

* the characteristics of the technical and organizational project, which is being evaluated, particularly with regard to complexity, novelty and size,

* time span of technology (or project) implementation or exploitation,

* the social and cultural characteristics of the users.

The above named features are much consistent with the ontology-aided technolo-
gies and solutions, which exhibit a high degree of such characteristics like: novelty,
complexity and coupling, openness, networking, changeability, intelligence [10].

3 Modification of integrated method of technical projects
assessment

Having outlined the most important rules of the evaluation process we can now
present the selected evaluation method that will comply as much as possible with the
mentioned determinants and rules. Such a method seems to be the modified integrated
method of efficiency evaluation of technical and organizational projects [5].

The integrated method has two main features, which encourage to adopt it, such as
complexity and the organized choice of measures according to the rules of modular-
ized structure. Module is understand as a distinctive element of system performing a
specific function, which may be interchangeable into one that fulfils the same func-
tion, but of different internal structure. The adoption of the module requires a precise
description of its coupling with other modules. The modular structure of the modified
integrated method of technical project evaluation is presented in Fig.2.

The modified integrated evaluation method follows the ontological approach, as it
complies with the rules of formal knowledge representation. Adopting such an idea
may be particularly useful for complex technical solutions, like ontology-aided solu-
tions [12]. The use of this method may also facilitate knowledge communication be-
tween humans and technical applications.

Regarding applications it can be characterized by the basic formal categories of on-
tologies, i.e.: classes/concepts, objects (subjects), relations, attributes (properties),
events, rules. The proposed evaluation method may facilitate the effective and effi-
cient information and knowledge processing. It may also act as a surrogate of very
wide and complicated knowledge. It has also a descriptive character. It may be useful
in the area of designing, implementation and exploitation of technical solutions, as
well as for technology, production and logistics evaluations [11].

Having presented the idea of modified integrated evaluation method of technology
efficiency or technical projects we can start its detailed description. The method con-
sists of six modules that comprise the specific sets of measures. It includes the follow-
ing modules:
<figure><img src="image_4.png" /><figcaption>Fig. 2. The structure of the modified integrated method of technology evaluation</figcaption></figure>

1. Fragmented (sectional) simple measures (MOP) – a set of measures that characterize the basic inputs to the production process i.e. technical and technological equipment, all kinds of materials and resources, human factor etc.

2. Fragmented (sectional) complex measures (MOZ) - a set of measures which characterize the relations between basic economic sizes typical for production, i.e. capital labour ratio, material intensity, labour intensity, efficiency, productivity etc.

3. Synthetic measures (MS) - a set of measures which characterize in a holistic manner the production process and its results, i.e. the costs in various cross sections, the financial result, various types of profits etc. They should pay particular attention to the level and character of the technical solution e.g. through diverse taxation of the financial result which, as a result, defines the profit level.

4. The preferential innovation (investment) measures (POI) - a set of measures that characterize (evaluate) innovative solution which has the form of an investment project, taking into account social and economic policy of the specific country, conveyed through tax burden, subsidies and grants, fiscal and monetary policy etc. The determinants of the size of the innovation measures (indicators) include e.g. interest rates, tax thresholds and rates, etc.

5. The module of technical and operational parameters (PT-U) - a set of measures that characterize production or service output applying for the description mostly performance parameters (usage) i.e. functionality, universality, durability, the cost of use etc. The relations and accidents happen according to previously observed rules among these typical ontological attributes.

The module of social and environmental parameters (PS-Ś) - a set of measures that characterize influence of product or service on some social and environmental parameters. Compliance with them should ensure the proper state of the environment in the future. Ontological approach is also adopted here, reflected in the observation of relations and accidents happening according to the previously observed rules, and a possible response to the ongoing changes, if they are not coherent with the desired condition of the society and environment.
Due to the topic of publication more attention is paid to PT-U and PS-Ś measures modules than others in the following part of the paper. In-depth analysis was devoted to the environment elements that are determined by the choice of measures and their later level, as well as the future dynamics expressed through the proper indicators. We may observe it, in reference to both modules, by detailed analysis of conditioning of modules operation (Fig.3).

<figure><img src="image_5.png" /><figcaption>Fig. 3. Operations conditioning of the module of technical and operational parameters PT-U and social and environmental parameters PS-Ś</figcaption></figure>

The structure of both modules of measures (PT-U and PS-Ś) may be subject to
change because of: ongoing technical and technological progress, the characteristic of
the evaluated products and services, from the perspective of the evaluated features,
the change of the desired condition of environmental, legal, cultural and social re-
quirements. In recent years, changes have been occurring in shorter time intervals.
Sometimes they have a turbulent character, which makes difficult to adopt them with-
in the operational evaluation system. As a result, it increases the risk of obtaining
unsatisfactory society or environment condition, due to the applied method.

4 Application of controlling in technology evaluation

The continuous application of PT-U and PS-Ś measures modules requires the use of controlling and its philosophy. It may be defined, adopting ontology rules, as the system of correlated rules, methods, techniques and projects that are intended for internal control system, aiming to obtain the desired outcome. Controlling may be also understood as an integrated subsystem of managing, planning, control and information, which supports the adaptation and coordination of the whole management system [6]. The idea of controlling has been presented by a number of authors in their works. However, the most precise presentation is by Kuc, who states that “controlling
as an integration platform of methods and management techniques, not only allows identifying quickly the place of unexpected changes and their reasons, but is also helpful when taking remedial measures. As a result of such possibilities, ideas appear such as management by controlling” [4].

In production management, if one adopts ontologies, it is necessary to: clearly determine strategic and operational goals of the organizational unit, plan general and specific tasks, and define relations which may occur between them during the goal realization, define the implementation methods. In order to achieve that, the modified integrated method of efficiency evaluation may be useful, as well as the adoption of controlling philosophy for the stage of realization of goals.

Having outlined the controlling idea and definition, one may present controlling tables prepared according to the modified combined method of efficiency evaluation regarding PT-U and PS-Ś modules (Tables 1 and 2).

The tables include the basic measures that belong to two groups: technical and operational, and social and environmental. They have been presented according to their peculiarities. They have a universal character.

In order to verify the functioning of the modified integrated method of efficiency evaluation of technical projects e.g. houses, modern, integrated production or logistic systems etc., it is necessary to continue practice oriented research that should constitute the next stage of research work.

<table><caption>Table 1. Example controlling setting of technical and operational measures (PT-U)</caption><thead><tr><th rowspan="3">Specification of the monitored measures</th><th colspan="4">The size (values) of measures/ the preferred timetable</th></tr><tr><th rowspan="2">Target values at the end of t period (year)</th><th colspan="4">Cumulated Values at the end of particular stages (e.g. quarterly sub periods)</th></tr><tr><th>t1</th><th>t2</th><th>t3</th><th>t4=t</th></tr></thead><tbody><tr><td colspan="6"><strong>1. Measures of technical parameters (design / production)</strong></td></tr><tr><td>1.1. Measure of material requirements</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>1.2. Size (shape) measure</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>1.3. Force (power) measure</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>1.4. Speedo</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>1.5. Compatibility</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td colspan="6"><strong>2. Measures of operational parameters (design / production)</strong></td></tr><tr><td>2.1. Exploitation measure</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>2.2. Productivity measure</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>2.3. Usability measure</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>2.4. Complementarity level</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>2.5. Durability</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td colspan="6"><strong>3. Measures of economic parameters expenditures and costs (production)</strong></td></tr><tr><td>3.1. The measure of minimum expenditures reg. the performance of technical functions</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>3.2. The measure of minimum expenditures reg. the performance of exploitation functions</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>3.3. The measure of minimum costs of operational function</td><td>+</td><td>+</td><td>+</td><td>+</td></tr></tbody></table>

Note: + defines the values, i.e. indicators (parameters)
<table><caption>**Table 2.** Example of controlling table of social and environmental measures module (PS-Ś)</caption><thead><tr><th rowspan="2">Specification of the monitored measures</th><th rowspan="2">Target values at the end of t period (year)</th><th colspan="4">The size (values) of measures/ the preferred timetable</th></tr><tr><th>t1</th><th>t2</th><th>t3</th><th>t4=t</th></tr></thead><tbody><tr><td colspan="6"><strong>1. The measures of social aspect of communication</strong></td></tr><tr><td>1.1. The value of investment in environmental protection</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td>1.2. The level of social awareness regarding social relations, environmental protection</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td>1.3. The level of Internet use (the average time of Internet use)</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td>1.4. The utilization rate of product capacities (e.g. mobile phones, computers etc.)</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td colspan="6"><strong>2. The measures of social aspect of development</strong></td></tr><tr><td>2.1. Availability and quality of public infrastructure</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td>2.2. GDP per capita in PPP</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>2.3. The level of public security</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>2.4. The amount of free time</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td>2.5. Availability and quality of cutting-edge durable goods</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td colspan="6"><strong>3. The measures of the level of healthcare</strong></td></tr><tr><td>3.1. The rate of disease (the number of new patients/ total number)</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td>3.2. Mortality rate (deaths/number of people)</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td>3.3. Mortality rate (deaths caused by disease/number of people)</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td>3.4. Prevalence rate (number of patients/ total number)</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td>3.5. Average lifespan</td><td>+</td><td></td><td></td><td></td><td>+</td></tr><tr><td colspan="6"><strong>4. The measures of nature condition</strong></td></tr><tr><td>4.1. Forest condition</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>4.2. Agricultural soil condition</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>4.3. Post-industrial devastation</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>4.4. The usage of nonrenewable resources</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td colspan="6"><strong>5. The measures of air quality</strong></td></tr><tr><td>5.1. Total dust</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>5.2. Sulphur dioxide (SO2)</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>5.3. Nitrogen oxides (Nox)</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td colspan="6"><strong>6. The measures of water quality</strong></td></tr><tr><td>6.1. Total nitrogen</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>6.2. Total phosphorus</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>6.3. Nitrates</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>6.4. Water oxygenation</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>6.5. The pH of water</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td colspan="6"><strong>7. The measures of waste management</strong></td></tr><tr><td>7.1. Plastic waste</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>7.2. Agricultural and food waste</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>7.3. Hospital waste</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>7.4. Industrial waste</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>7.5. The rate of waste processing</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>7.6. The rate of waste utilization</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr></tbody></table>
# 5 Conclusions

To resume, it must be noted that the significance of the topic is quickly growing. It results from high intensity of implementation of novel technologies, products and services, in the area of ICT, biomechanics, communication etc. The evaluation method is based on ontological approach and adopts the paradigm that includes the balance of economy, society and environment. As a complex and universal method it should be elaborated and implemented as fast as possible. A research regarding its functioning in a representative context, should be conducted.

**Acknowledgement:** This work has been co-funded by the PSP 504/02105/1103/40.000105 grant of Warsaw University of Technology and by the EU ARTEMIS project no. 332946 "ESCOP - Embedded systems for Service-based Control of Open manufacturing and Process automation". The author thanks to partners of the projects.

# References

1. Abdel-Kader, M.G., Dugdale, D.: Evaluating investments in advanced manufacturing technology: A fuzzy set theory approach. British Accounting Review, 33, 455-489 (2001)

2. Głodziński E., Marciniak S., The role of economic information in managing of technical projects (in Polish). In: Information resources in enterprise management. Dom Organizatora, Toruń, pp. 191-208 (2011)

3. Goyal, S., Grover, S.: Advanced manufacturing technology effectiveness: A review of literature and some issues. J. Frontiers of Mechanical Engineering, vol.7(3), 256-267 (2012)

4. Kuc, B.R.: Controlling as a tool for early warning (in Polish). PTM Publishing House, Warsaw, (2004)

5. Marciniak, S.: The Combined Method of Efficiency Evaluation of Technical and Organizational Projects (in Polish). Warsaw University of Technology Publishing House, Warsaw (1989)

6. Marciniak, S.: Controlling. Theory. Application (in Polish). DIFIN, Warsaw (2008).

7. Mohanty, R.P., Deshmukh, S.G.: Advanced manufacturing technology selection: a strategic model for learning and evaluation. Int. J. of Production Economics, 55, pp. 295-307 (1998)

8. Sarkis, J.: Evaluating flexible manufacturing systems alternatives using data envelopment analysis. In: The Engineering Economist: A Journal Devoted to Problems of Capital Investment, vol.43, pp. 25-47 (1997)

9. Weber, J., Zubler, S.: Controlling in Zeiten der Krise. Advanced Controlling, vol.73, pp. 47-52 (2010)

10. Strzelczak, S.: Ontology-Aided Management. Silesian University of Technology Series in Management, No.73, pp. 619-630 (2014)

11. Garetti, M., & Fumagalli, L.: P-PSO ontology for manufacturing systems. In: Proceedings of the 14th IFAC Symposium on Information Control Problems in Manufacturing, pp. 247-254 (2012)

12. Lobov, A., Ubis Lopez, F., Villasenor Herrera, V., Puttonen, J., & Martinez Lastra, J.L.: Semantic Web Services Framework for Manufacturing Industries. In: Proceedings of the IEEE International Conference on Robotics and Biomimetics (2009), 2104-2108 (2009)