Source: http://gerpisa.org/node/4972
Timestamp: 2019-04-25 19:47:02+00:00

Document:
The autonomous vehicles (AV) have the potential to completely change the automotive market. This technology will enable reduction of accidents, pollution, energy consumption and costs associated with traffic. Furthermore, reports indicate that the autonomous vehicles will be widely adopted already in 2020. Although the application of this technology has innumerable advantages, ethical aspects like liability, AV moral decisions and social aspects like the effect on employability must be considered when designing, testing and validating these systems. Most of the testing and validation on AV today follow the classic V model, when the test cases (set of inputs and their expected outputs) derive only from technical design requirements. This could let important requirements out of the analyses and lead to significant design and test problems. In this context, the philosophy of technology can provide important tools to analyze, to have a critical view and point a direction to technology. In particular, the Dutch school of philosophy of technology has an interesting conceptual framework. The Theory of Modal Aspects (TMA) states that every entity (including technological artifacts) exists in the reality in multiple aspects. Not only technical aspects (arithmetic, spatial, kinematic, physical), but also non-technical aspects usually neglected by engineers (social, economic, juridical, moral, among others), are considered in the analysis of the entity. Second, the Theory of Structures of Individuality (TSI). In it, three functions of a technological artefact are defined to show its identity. First, the foundational function is defined, that provide which modal aspect gives the origin of the technological device. Second, the qualifying function is defined, that provide which aspect gives the context of application of the device under analyzes. Then, the operational function is defined, that provide which modal aspect gives the operation of artefact. With the origin, context of application and the operation of the AV well defined, it is possible to better understand its identity. Therefore, this paper has the objective to identify new challenges imposed by the context of autonomous to several steps in the V model. Each new challenge is compared with the possibilities derived from the philosophy of technology, in particular with theory of modal aspects and the theory of structures of individuality, identifying opportunities to propose an extended V model that would include non-technical requirements. The methodology steps are as follows. Identify an appropriate V model. Enumerate aspects in the scientific literature about the challenges of AV validation. Allocate the challenges along the steps of the V model. Identify applications of the TMA and the TSI to similar design problems. Associate the TMA and TSI contributions with the steps of the V model. Propose research action that would lead to an extended V model in near future. The preliminary results show that there is a technical bias in the designing, testing and validating of autonomous vehicles. Non-technical aspects are neglected by designers and engineers, which in turn could generate important project problems. They also show that philosophy of technology, more specifically the Dutch school of philosophy of technology, can provide important tools for analyzing AV highlighting its complexities. This analysis can lead to an extended V model with a more holistic view of the autonomous vehicles validation. The extended V model may help engineers to define more accurately all of the requirements involving the autonomous vehicles. It may also help policy makers to better propose legal requirements and standards to work as guidelines in their specific political contexts. This could help avoid design or test problems, delivering a more efficient and save product to its final client.
According to Nilsson (2010), Artificial Intelligence (AI) can be defined as an activity involved in making intelligent machines and intelligence can be understood as the capacity of an entity to work properly and with foresight in its environment. The definition given by Nilsson is very generic, but necessary to approach several AI techniques and the contexts in which they are applied. AI applications are already a reality in many economic and industrial sectors and they have a big capacity to generate radical innovations (AZZAN; KHALIL; SAMI, 2017). These innovations can be very challenging to technology developers, users, managers and policy makers.
A recent study performed in Stanford University (STONE et al., 2016), identifies eight main AI application fields: (i) transportation, (ii) robotics, (iii) health, (iv) education, (v) low-income communities, (vi) public safety, (vii) employment and (viii) entertainment. The main technological trends in AI were also identified. Concerning machine learning, large-scale machine learning and deep learning are highlighted. Both techniques extensively use neural networks for pattern recognition. Another important machine learning technique is the reinforced learning, which is focused in sequential decision-making based on experience. It could also be highlighted the computer vison, robotics, natural language processing, internet of things, neuronal computation, among others. In 2016 the big technology companies (e. g. Google and Baidu) invested between 26 and 32 billion dollars in AI; investments in AI startups were between six and nine billion dollars; and the market forecast for AI in 2025 can be as big as 126 billion dollars (BUGHIN et al., 2017).
In spite of the fact that AI has the potential to improve humans’ life in all areas cited above, equally important is to seriously consider some social and ethical factors involved in the application of these techniques. Privacy and human labor substitution are good examples of issues that claim for deeper debate (STONE et al., 2016). Furthermore, some technical barriers – such reliable hardware for transportation and robotics or the interaction with experts in healthcare and education – must be taken into consideration in application of such technologies.
Among the main AI applications, autonomous vehicles (AV) stand out in the field of transportation. This technology has the potential of completely change the transportation market, reduce accidents, pollution, energy consumption and traffic costs (ANDERSON et al., 2016). Additionally, reports indicate that AV will be largely adopted in 2020 (STONE et al., 2016). On the other hand, the use of AV could reduce important incomes of government institutions related with tickets for traffic law violation and taxing of fossil fuels and CO2 emission (AZZAN; KHALIL; SAMI, 2017). Many jobs may come to disappear e.g. taxi, bus and truck drives, causing a significant social impact (ANDERSON et al., 2016).
Other ethical and moral questions can also be raised. For example, in a case of unavoidable accident, the decision of an AV of sparing the life of the passengers or pedestrians in the street. Bonnefon, Shariff and Rahwan (2016) performed a study on this problem and concluded that most people would agree that an AV should spare the pedestrians when their number is much higher than the number of passengers. In contrast, most people would be less willing to buy an AV designed with this type of logic. Moreover, who would be legally responsible in case of an accident? The manufacturer? The passenger? Hevelke and Nida-Rümelin (2015) approached this subject through an ethical point of view. According to them, the passenger only could be considered responsible if there is a chance of any intervention. Additionally, if we consider the manufacturer and developers as the legally responsible parties, the AV technology could be improved, making them safer.
The problem can be also be approached from the perspective of the engineers and developers. An important activity of any engineering project is the test and validation, when it is verified if the developed systems comply with the design requirements and work as predicted. AV validation has an intrinsic and special challenge, since it is hard to determine the reason why a machine-learning algorithm take a particular choice (BOSTROM; YUDKOWSKY, 2014). This could cause problems related to error identification and correction. The validation process of an AV can be explained by a V model (MASUDA, 2017; HUANG et al., 2016), in which the technical artefact is specified, designed and tested using only functional requirements as reference. Because of that, studies on AV testing and validation (WOLSCHKE et al., 2017; LI et al., 2017; MASUDA, 2017; HUANG et al., 2016) always have a strong technical bias. In general social, ethical and fiduciary aspects are not considered, which could exclude important requirements from the analysis. With regard to the product innovation management, the lack of clear requirements could jeopardize the development and application of AVs. A good example of this is the recent approved laws in US for the use of AVs. While the Nevada State law has only 23 lines, the Californian has 6 pages (FAGNANT; KOCKELMAN, 2015). This regulatory uncertainty might mean reworks and bigger costs for the manufacturers.
Besides that, studies on philosophical, ethical and social aspects related to AVs (BONNEFON; SHARIFF; RAHWAN, 2016; HEVELKE; NIDA-RÜMELIN, 2015) generally tend get far from engineering world. This creates difficulties to bring such concepts to the engineering practice. The present study intends to reduce such distance and to pave the way towards an extended V model, wherein both technical and philosophical aspects are considered when designing, testing and validating AVs. A research program is proposed to explore the dialogue possibilities between the AV validation theory and the philosophy of technology. The methodology steps are: (i) detailing the main aspects of AV validation using the V model; (ii) performing a literature review of the philosophy of technology, focusing on the Dutch school of Philosophy of Technology, and particularly on the Theory of Modal Aspects (TMA) and Theory of Structures of Individuality (TSI); (iii) identify opportunities to associate the TMA and TSI contributions with the steps of the V model; and lastly (iv) proposing a research agenda towards an extended V model in near future.
The organization of this paper follows the proposed steps. In section 2, the main aspects concerning AVs technology are presented. In section 3, the V model and the AV validation process are discussed. Section 4 presents a literature review of philosophy of technology. In Section 5 the dialogue opportunities are identified and explored, heading the paper to the final considerations which includes the proposition of a research agenda that could lead to the Extended V model.
According to Anderson et al. (2016), it is possible to see a continuum between vehicles completely operated by humans and vehicles totally automated. The National Highway Traffic Safety Administration (NHTSA) expressed this continuum in a five-level scale described as follows (PRELIMINARY…, 2013).
•	Level 0: there is no automation, the driver is completely and solely responsible for both controlling the vehicles primary functions (break, steer, etc.) and monitoring the traffic conditions, with the objective to safely operate the vehicle.
•	Level 1: in this level, there is an automation of one specific function, or more than one function working independently. The driver is still solely responsible for the safe operation of the vehicle, but can chose to give authority to the system in specific cases. The system can also take limited control of a primary function, or help the driver in normal situation or in imminent accidents.
•	Leve 2: in this level, at least two primary functions are automated, working jointly with one another, in a way to free the conductor from the responsibility over these functions. The conductor is still responsible for the safe operation of the vehicle and must be ready to take control, if necessary.
•	Level 3: this level allows the conductor to deliver total control of critical functions to the system, under certain traffic and environment conditions. It is expected that the conductor take control of the vehicle, if necessary, with a comfortable transition time.
•	Level 4: the vehicle is designed to control all critical control functions during the entire trajectory. The passenger is only responsible to give the destination information and it is not expected that he/she will take control during any moment of the trip.
Huang et al. (2016) proposed a general architecture to level 4 autonomous vehicles (Figure 1). The perception layer is responsible for data acquisition from the several sensors present in the vehicle. The decision layer sends feedback information used to optimize the perception layer, furthermore it interprets the data in order generate a reasonable output to the action layer. The artificial intelligence algorithms are located in this layer. The action layer, then, receives abstract data from the decision layer and transforms them into set points to the actuators.
Concerning the tests and validation, each layer has its particularities. According to Huang et al. (2016), the AV validation can be divided into four phases. The first phase is software testing. The software in the VA has millions of code lines, therefore automated tests are necessary in this phase. Figure 2 shows the workflow of an automated software test (MASUDA, 2017). In test design, the overall test characteristics are defined. All test cases, the sequence of inputs and the expected outputs, are then selected. Finally, the test are performed and the results are generated.
The next phase is the simulation tests, wherein high-fidelity simulations are used to test the AV systems. Li et al. (2017) model the multiple interaction among the drivers in traffic using game theory. The following phase is denominated “X-in-the-loop”. In this phase, simulation techniques are applied to the actual sensors and actuators of an AV in systems, called “Hardware-in-the-loop” (HIL) and “Vehicle-in-the-loop” (VEHIL).
The last phase of the AV validation consists of driving tests in real traffic situations. In this phase, the AVs are tested in real cities or in test centers like the M-City and the iVPC. These centers have several test environments, such as urbans areas, rural areas, high velocity areas, among others. The big advantage of this kind of center is its capacity of performing the tests in a structured and repetitive way. According to Huang et al. (2016) and Masuda (2017), a V Model can summarize the whole validation procedure.
Figure 2 – Workflow of AV software testing.
The V Model The V model is a process life cycle model that uses sequential approach to visualize several key areas for the concept, development and validation of a product (WALDEN et al., 2015). It “…highlights the need for continuous validation with the stakeholders, the need to define verification plans during requirements development, and the importance of continuous risk and opportunity assessment” (WALDEN et al., 2015 p 33). Descending the V, through stakeholders needs and functional requirements, a concept of a system is created and the elements comprising that system are identified (Figure 3). This phase is called “Architecture decomposition and definition”. At the base of the V, each system element is developed and realized. Going up in the V, there is the phase called “Architecture integration and validation”. In it, every aspect of the system is tested and validated according to the requirements of each level (lower level, upper level, system or solution level).
Adapted from Walden et al. (2015 p. 34).
The descended and ascended parts of the model can be described in Figure 3. The usual questions addressed in the Architecture decomposition and definition phase are as follows. Are the proposed baselines acceptable? This question concerns the stakeholders’ needs and functional requirements. How to prove it´s built right? This question will define the integration, verification and validation plans. How are the opportunities and risks of the proposed baselines being resolved? This question involves risk management and the actions required in case of problems in the design and testing of a product. There are also some common questions in the Architecture integration and verification phase. Every test of each level of the V model tries to answer the question is the verified performance acceptable? For problem investigation and resolution, the question is the problem cause understood? is addressed. The V model is a very useful management tool wherein every development decision is linked to a stakeholder need and a functional requirement and is verified and validated according to a specific plan.
Figure 4 describes a V model applied to AV development and validation. The previously discussed four phases of AV validation can be placed in the different levels of the V model. Software testing is near the base of the V since it is closely related to the specification of modules and system elements. The X-in-the-loop phase is located throughout all the V model levels, since it comprises the hardware of the whole system. The simulation and driving tests are in the top of the V, where the entire system is validated according to specified functional requirements.
Figure 4 – V model application for AV.
From other perspective, Wolschke et al. (2017) affirms that the AV requirements are very complex and, because of that, it is difficult to properly apply them in a classic V model. He defends an approach based in observation, more focused in behaviors. His definition of behavior is the reaction to certain situations. Therefore, the test cases are designed according to real and simulated traffic data and not through functional requirements. Although the Wolschke´s proposal is relevant to the debate, the problem is still approached from a technical perspective. As usual, social, ethical and fiduciary aspects are not under consideration. A good dialogue with the philosophy of technology can be useful to enrich the discussion and to pave new ways for solutions.
According to Verkerk et al. (2015) the philosophy of technology (PoT) has three main objectives. The first is the analytical function, in which the philosophy is used to create a conceptual framework to make the discussion possible. In a second instance, the philosophy has the function of criticizing the role of technology in the society, which is only possible under a good conceptual framework. Finally, the philosophy has the role of pointing to a direction. Here, important questions guide the reflection: what is a good technological apparatus? How the technology influences the human beings and the society? These three functions of philosophy are of fundamental importance to engineering.
Philosophical reflections on technology have been done since the ancient Greece when Plato described the world as the work of a craftsman and when Aristotle made the distinction between natural things (physis) and artefacts (poesis). Despite that, the first book specifically dedicated to the philosophy of technology was published only in the second half of the nineteenth century (KAPP, 1877). The end of the nineteenth century is considered a milestone in the development of the philosophy of technology, since it was the first time that the technological phenomenon was viewed as an interesting object in itself, for philosophical inquire – and not as a part of a general philosophical project.
The family of philosophical reflections published in the beginning of the twentieth century until its penultimate decade was known as the “classical” PoT (e. g. Brey, 2010). Because many of its representatives had a background in social sciences and focused their work on the relationship between technology and society (rather than on technology itself), this tradition was also known as “Humanities Philosophy of Technology” (MITCHAM, 1994). Under this perspective, technology is viewed as a culture product (permeated by values) and the focus is on its negative effects in humanity and society. Therefore, the humanities PoT was established in a context of the consolidation of a technological society and shed light on its harm to the human condition and to the culture; thus it is marked by a fundamental pessimism in relation to technology.
By the end of the 1980s, the classical PoT began to be criticized from several perspectives. Firstly, after almost half a century since the end of World War II and two decades since the peak of the counterculture movements of the sixties, the collective disposition for the technological phenomena was changing from an unbalanced pessimism to a more ambivalent notion, that saw technology as the promoter of both goods and evils. In parallel, in the academic realm, the field of science and technology studies (STS) emerged. It emphasized the necessity of an empirical orientation in the investigation of scientific and technologic processes, highlighting its socially constructed character (SISMONDO, 2003). These studies contributed to the questioning of the deterministic classical image of technology as a monolithic and unescapable meta-phenomenon. These generalizations, characteristics of the state of the PoT so far, were replaced by attempts to locally describe the relation between concrete technological processes and their specific historical situation. This was called the empirical turn in the philosophy of technology (PITT, 1995; KROES; MEIJERS, 2000; ACHTERHUIS, 2001).
An important unfolding of this empirical turn is called the “Engineering Philosophy of Technology” (MITCHAM, 1994). This movement began in the last decade of the twentieth century and in the beginning of the 2000s and it not only had an empirical emphasis, but also an analytical one. Its main interests falls over rigorous concept definitions and the logical relation among them for the description – minimally evaluative or prescriptive – of technological practices (i. e. methodological analysis), of the knowledges involved (i. e. epistemological analysis) and of the resulting artefacts (i. e. ontological analysis). In this sense, it is a radically different approach in regard to the classical PoT and get closer to the classical tradition of philosophy of science. Together with other manifestations of the empirical turn, the engineering PoT and its analytical tradition has become the dominant perspective in the last 25 years in different countries and motivating several research contributions.
The empirical and analytical emphasis of the engineering PoT is very useful in the analyzes of AVs and of the V model application for design and validation of such technology. Especially the empirical-analytic emphasis of the Dutch school of Phylosophy developed by authors like Maarten Verkerk (VERKERK et al., 2015) and Andrew Basden (BASDEN, 2002; BASDEN; BURKE, 2004; BASDEN; WOOD-HARPER, 2006; BASDEN, 2011). Significant part of their work is based on the Theory of Modal Aspects (TMA), on the Theory of Individuality Structures (TSI), on The Theory of Ground Motives and on the New Critique of Theoretical Thought (NCTT), originally proposed by the Dutch philosopher Herman Dooyeweerd (DOOYEWEERD, 2016).
d)	Physical, because they take on energy levels – and thus mass.
o)	Faith, because they may serve the beliefs and aspirations.
From the numerical to the biotic aspects, the technology operation is submitted to unescapable laws. By contrast, in the posterior aspects, their appropriated openness to meaning is dependent, not on laws, but on positivation of norms that may be transgressed (or not) by humans. Nevertheless, every aspect is governed by its own legal-normative nucleus, establishing a specific sovereign sphere, irreducible to the rest. Despite this irreducibility, the aspects operate under an ontological interdependency in the listed order – i. e. from “a)” to “o)”. In other words, to occupy space, an individuality structure needs to present quantities; to make movements, it needs to occupy space; to take on energy, it needs to make movements; and so forth.
In addition to establish the characteristics of a technological artefact in all aspects of reality, this conceptual framework makes possible to determine the artefact´s identity. Three concepts are used for this. The first concept is the foundational function of the dispositive that points which aspect best responds for the origin of the dispositive. The second is the qualifying function that points to the aspect that responds to the context in which the dispositive is applied. Finally, the operational function points to the aspect in which the technological dispositive operates. Since the technological artefacts are products of processes characterized by a controlled formation, they are always founded in the formative aspect and execute their operation on the subjacent aspects. In this way of thinking, the technology, as a human cultural activity, would involve socio-material practices of development and the use of physical artefacts to clarify meaning. That is why, in the Dutch philosophy tradition, the technology tends to be defined as “search for meaning” (VERKERK et al., 2015).
Another implication of the Dutch school of philosophy is the proposition to understand the technology as a fundamental human relation. Since this relation happens, in a concrete way, in the totality of human life, it also reflects the ground motives that guide their existence and their transcendental idea of origin (archê) of their temporal existence. This way, the Dutch school points to the insufficiency of the critique made by the predominant immanent philosophies (i. e. rationalists), since they would not be capable of localizing the relation with the technology in a scope of pre-theoretical experience – and thus they end up affirming the dogma that the source of the thought on technology is the reason itself. As a consequence, this techno-science culture imposes a technicist and rationalist normativity to the society, obsessed by efficiency, effectiveness and logic – which, going against the original search for meaning, impoverishes the broad diversity of life meanings in its human, social and environmental dimensions.
This philosophical tradition and its application to technology provides many opportunities to see the technical artefacts from another perspective. It also provide a possibility to enrich the V model concept avoiding any reductionism and technical bias. The next section presents these dialogue opportunities.
As stated before, the V model is an important management tool to the development of new vehicles. It helps to propose a design, even for the lower level system elements, that is linked to the functional requirements established upon the stakeholders needs. It also allows the developers to make a validation plan to test the functionalities from the lower system levels to the whole system realization. The problem is, when it comes to AI in general and AV in particular, the design decisions and, by consequence, the test and change decisions involve more than functional requirements. In this context, the philosophy of technology, especially the empirical-analytic emphasis of Dutch school, can provide good insights for a more holistic management decisions that would include other complexities involved with the technology. As shown in Figure 5, the design and test decisions are permeated by the analyzes of the ground motives, of the modal aspects and of the functions of designed application. Each part is explored separately as follows.
At the top of the V, before any design decision is made or any functional requirement is considered, a reflection on the ground motives subjacent to that technological development can be made. Some questions could guide this reflection. What are the beliefs and aspirations that inspire this technological development? Are these motivations in conflict with a stakeholder need? How far are these motivations malefic to society? How far are these motivations benefic to society? That reflection would avoid the rationalist normativity of the techno-science culture and regain the notion of the technological development as the disclosure of meanings. The analysis could go beyond efficiency, effectiveness and logic and the real threats and opportunities could be clarified.
Figure 5 – Dialogue opportunities between the V model and the philosophy of technology.
The modal aspects analysis could enrich every step of the design flow. Every functional requirement would be considered in light of the modal aspects and this logic would follow the steps down in the model. The emphasis here would be in the aspects in which the artefacts work as objects, since their legal-normative nucleus could be violated by human activity. Some questions could also guide this reflection. What are the most relevant modal aspects for this project? Are all of the relevant modal aspects being considered in the design decisions? Are any legal-normative nucleus being transgressed by the design decisions? The technical bias could give way to human-social aspects (moral, lingual, social, psychic), integrating them to the engineering decisions throughout product development.
The operational and qualifying function can enrich the module construction phase at the base of the V. As stated before, the operational function concerns with what the artefact actually does and the qualifying function with the context in what that artefact perform its operation. In the module construction phase, the first concern is the operational function (VERKERK et al., 2015). The main concern here is technical with the identification of which aspect gives the artefact´s operational function and will guide its design. Then, further development of the artefact can take place and the design decision will focus on how to improve the dispositive performance to a specific context (qualifying function). Poor considerations on the operational function can cause functional problems and poor consideration on the qualifying function can make the artefact less user-friendly. Therefore, this analyzes can contribute to better designs.
The modal aspects analysis can also improve the validation flow. Here, the decision of compliant and not-compliant are made. The sum of all these decisions will compile a product that will be massively produced and delivered to its final client. The modal aspects analysis can help the validation flow in two different phases. First, in the test planning and the establishment of the compliant and not-compliant references. A holistic view of all of the aspects involving a technological development would help design tests that are more accurate in representing reality. Second, every not-compliant decision will generate a change in the project. These changes need also be according to the legal-normative nucleus of the relevant aspects.
All these discussion may contribute to the proposition of a new technological management tool. A tool that would enable a holistic view of the technological apparatus and of the design efforts.
This paper explored the dialogue possibilities between the V model and the philosophy of technology. The proposed interaction has two main motivations. First AI technologies with the capacity to make autonomous decision involving human lives, like AVs, tend to address complex philosophical and ethical questions. In addition, the application may have influence on serious social issues like labor substitution and govern revenues. Second, it is very hard to intentionally apply philosophical concepts in the engineer practices, since engineers tend to approach the technological issues only from a technical point of view. This bias could let important requirements out of analysis and lead to design problems. The V model application for the design and validation of AVs was presented, followed by the main aspects of the Dutch tradition of PoT and the dialogue opportunities. It was demonstrated that PoT could enrich the discussion in the design flow, the module construction and the validation flow.
All of the dialogue opportunities could lead to research actions to propose a new technology management tool, the Extended V model. For future research efforts, it is necessary to prescribe and map actual processes wherein the dialogue would happen in a way that is applicable for engineering organizations. It is also necessary to present the proposed model to leaderships in the engineering organizations and use it in real design projects to verify its applicability.
The dialogue possibilities between technology management tools like the V model and philosophical traditions are enormous. Since humans are responsible for the technological development and since technologies like AI and AV are growing in both application and complexity, this dialogue becomes more than an opportunity it becomes a necessity.
ACHTERHUIS, H., Ed. American Philosophy of Technology: The Empirical Turn. Bloomington: Indiana University Pressed. 2001.
ANDERSON, J. M. et al. Autonomous Vehicle Technology: A guide for policymakers. Santa Monica: RAND Corporations, 2016.
AZZAN, M.; KHALIL, T.; SAMI, N. Technology disruption for development and peace. In: IAMOT 2017 Conference Proceedings. [S.l.: s.n.], 2017.
BASDEN, A. The critical theory of Herman Dooyeweerd? Journal of Information Technology, v. 17, n. 4, p. 257-269, Dec 2002.
______. Enabling a Kleinian integration of interpretivist and socio-critical IS research: the contribution of Dooyeweerd's philosophy. European Journal of Information Systems, v. 20, n. 4, p. 477-489, Jul 2011.
BASDEN, A.; BURKE, M.E. Towards a philosophical understanding of documentation: a Dooyeweerdian framework. Journal of Documentation, v. 60, n. 4, p. 352-370, 2004.
BASDEN, A.; WOOD-HARPER, A.T. A philosophical discussion of the root definition in soft systems thinking: An enrichment of CATWOE. Systems Research and Behavioral Science, v. 23, n. 1, p. 61-87, Jan-Feb 2006.
BONNEFON, J.-F.; SHARIFF, A.; RAHWAN, I. The social dilemma of autonomous vehicles. Science, American Association for the Advancement of Science, v. 352, n. 6293, p. 1573–1576, 2016. ISSN 0036-8075.
BOSTROM, N.; YUDKOWSKY, E. The ethics of artificial intelligence. In: . The Cambridge Handbook of Artificial Intelligence. [S.l.]: Cambridge University Press, 2014. p. 316–334.
BREY, P. Philosophy of Technology after the Empirical Turn. Techné: Research in Philosophy and Technology, v. 14, n. 1, 2010.
BUGHIN, J. et al. ARTIFICIAL INTELLIGENCE THE NEXT DIGITAL FRONTIER? [S.l.], 2017.
DOOYEWEERD, H. A New Critique of Theoretical Thought. Paideia Press, 2016.
FAGNANT, D.; KOCKELMAN, K. Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations. Transportation Research, Part A: Policy and Practice, Elsevier Limited, v. 77, p. 167–181, 7 2015.
HEVELKE, A.; NIDA-RÜMELIN, J. Responsibility for crashes of autonomous vehicles: An ethical analysis. Science and Engineering Ethics, v. 21, n. 3, p. 619–630, Jun 2015. ISSN 1471-5546.
HUANG, W. et al. Autonomous vehicles testing methods review. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). [S.l.: s.n.], 2016. p. 163–168.
LI, N. et al. Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems. IEEE Transactions on Control Systems Technology, PP, n. 99, p. 1–16, 2017. ISSN 1063-6536.
KAPP, E. Grundlinien einer Philosophie der Technik: Zur Entstehungsgeschichte der Cultur aus neuen Gesichtspunkten. Braunschweig: G. Westermann, 1877.
KROES, P.; MEIJERS, A.W.M., Eds. The Empirical Turn in the Philosophy of Technology. Amsterdam: JAIed. 2000.
MASUDA, S. Software testing design techniques used in automated vehicle simulations. In: 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). [S.l.: s.n.], 2017. p. 300–303.
MITCHAM, C. Thinking Through Technology: The Path between Engineering and Philosophy. Chicago & London: University of Chicago Press, 1994.
NILSSON, N. J. THE QUEST FOR ARTIFICIAL INTELLIGENCE: A history of ideas and achievements. New York: Cambridge University Press, 2010. PITT, J., Ed. New Directions in the Philosophy of Technology. Dordrecht: Kluwered. 1995.
SISMONDO, S. An Introduction to Science and Technology Studies. Wiley-Blackwell, 2003.
STONE, P. et al. Artificial Intelligence and Life in 2030. One Hundred Year Study on Artificial Intelligence: Report of the 2015-2016 study panel. [S.l.], 2016.
VERKERK, M. et al. Philosophy of Technology: An Introduction for Technology and Business Students. Taylor & Francis, 2015. ISBN 9781317445708.
WALDEN, David D., et al. Systems engineering handbook : a guide for system life cycle processes and activities. Hoboken, New Jersey: Wiley, 2015.
WOLSCHKE, C. et al. Observation based creation of minimal test suites for autonomous vehicles. In: 2017 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). [S.l.: s.n.], 2017. p. 294–301.

References: V. 
 v. 
 v. 
 v. 
 v. 
 v. 
 v. 
 v. 
 v.