# On Affordances and Their Entailment for Autonomous Robotic Systems

Mihai Andries, Lorenzo Jamone, Justus H. Piater, Erol Sahin

► To cite this version:

Mihai Andries, Lorenzo Jamone, Justus H. Piater, Erol Sahin. On Affordances and Their Entailment for Autonomous Robotic Systems. Madhur Mangalam; Alen Hajnal; Damian G. Kelty-Stephen. The Modern Legacy of Gibson's Affordances for the Sciences of Organisms, Routledge, 2024, 9781003396536. hal-05249564

HAL Id: hal-05249564

https://imt-atlantique.hal.science/hal-05249564v1

Submitted on 11 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# ON AFFORDANCES AND THEIR ENTAILMENT FOR AUTONOMOUS ROBOTIC SYSTEMS

**Mihai Andries**

Department of Computer Science
IMT Atlantique
Brest, 29280 Plouzané, France

mihai.andries@imt-atlantique.fr

https://orcid.org/0000-0002-4047-314X

**Lorenzo Jamone**

School of Engineering and Materials Science
Queen Mary University of London
London E1 4NS, United Kingdom

l.jamone@qmul.ac.uk

https://orcid.org/0000-0002-1521-6168

**Justus H. Piater**

Department of Computer Science
University of Innsbruck
Innsbruck 6020, Austria

justus.piater@uibk.ac.at

https://orcid.org/0000-0002-1898-3362

**Erol Sahin**

Department of Computer Engineering
Middle East Technical University

Ankara 06800, Turkey
erol@metu.edu.tr

https://orcid.org/0000-0002-2993-6879

17 July 2023

## ABSTRACT

In robotics, affordances are used as a theoretical framework for guiding a robot's interaction with the objects within its environment. In particular, affordances are formalized in order to become compatible with machine learning approaches, which endow robots with autonomous capabilities. In this chapter, we will explore the role affordances play in cognitive robots' perception, such as in semantic mapping of the environment. We will see that, in this context, the aspect of direct perception seem irrelevant, as robots always employ a step for processing sensorial data. We will see how affordances interact with intentions, and how their most promising use currently seems to be in robotic action planning. Cognitive robotics, the branch of robotics aiming to endow robots with high-level cognitive abilities, is intensely exploring affordances as a pathway to robotic autonomy.

In robotics, affordances endow robots with autonomy by allowing them to identify action possibilities in their environment and predict their effects, making action planning possible. The robotics community is transitioning from a perspective where affordances (and robotic abilities, in general) are considered fixed (innate) (Myers et al., 2015; Do et al., 2018) o a perspective where affordances are considered learnable (acquired) (Montesano et al., 2008; Chavez-Garcia et al., 2017; Akbulut and Ugur, 2020). This learnable approach is championed by cognitive robotics in general, and developmental robotics in particular (Min et al., 2016). Here, robotic autonomy involves discovering and learning new affordances through sensorimotor exploration. In this perspective, when a robot is presented with an object, affordances allow us to understand how it can be used, that is, which functions it can provide. Thus, a robot is expected to recognize affordances available to it as well as to other agents in the environment. A robot aware of its perceptual/cognitive/actuation limitations (for instance, not being able to open a door), in a human-robot interaction setting, may ask for a service from a human who is able to provide it (Rosenthal et al., 2010).

## What do we understand by the term *affordance*?

At its core, the notion of affordance promotes the idea that an agent's perception abilities are shaped by its own action abilities, rather than being action-agnostic and generic. Coupled with the goal of the agent, this perspective
allows the agent to perceive only the properties of the environment that are relevant for actions towards achieving its goal. Meanwhile, the same environment may offer different action possibilities to different agents, depending on the sensorimotor capabilities of the specific agent. For example, a stable surface affords traversing to an agent that is able to locomote and whose body dimensions fit the size of the surface borders; a stone affords hammering to an agent who is able to pick it and who has enough force to lift it. Formally, this entails representing affordances as a relationship between an object and an agent, via an action, a resulting effect, all happening within an environment allowing this interaction. Describing objects, agents, actions, effects, and environments using their features (physical or logical) allows generic descriptions through pattern matching.

**What role does affordance play in perception? Is it the entity organisms perceive or the means through which organisms perceive? Are affordances the only perceptual dependent variables?**

**Role of affordances in perception.** Perception is the process of making sense of sensory signals. Affordances are entities that organisms perceive and discover through interaction with their environment. Although affordances have traditionally been associated to visual perception (Gibson, 1966), the concept of affordances can be associated to any of the other senses.

**Are affordances means or ends of perception?** Making this distinction may not be particularly useful, since perception is not generally a unidirectional processing pipeline. At least in higher animals, perception at its core involves reconciling an internal model with sensory signals. Nevertheless, organizing the perceptual world into affordances recruits them as means (in addition to ends) of perception. Perception gives rise to behavior, and information currently not perceived is filled in by memory or inference. Internal models can be placed along a continuum, from highly-reactive visuomotor behavior (like when playing tennis) to remarkably detailed internal models (like those of blind people). This coexistence of different processing scales has historically inhibited discourse by falsely pitting reactive against reflective behavior.

**Perceptual economy.** The means through which an agent perceives affordances are those of Ecological Perception, powerfully illustrated by Eleanor J. Gibson (1994; 2000; 2003): "*narrowing down from a vast manifold of (perceptual) information to the minimal, optimal information that specifies the affordance of an event, object, or layout.*" For this perceptual process to take place, the agent requires previous learning, that is, learning what is the minimal information to be picked: this happens both through evolution and development, leveraging the sensorimotor exploration of the environment by physical interaction.

The notion of affordance promotes *perceptual economy* by computing only the perceptual cues that are relevant to the agent's goals and action capabilities. Otherwise, action-agnostic perception would require an infinite amount of the computational power. Through such minimal perception, it becomes possible to link the perceptual computation to the actions of the agent. Such minimal and action-linked perception is usually referred to as direct perception. Coupling this linking with the predicted effect of the action is usually referred to as perceiving the world through the action possibilities afforded by the environment.

**Are affordances the entity organisms perceive, or the means through which organisms perceive?** If affordances were the only (primitive) means through which organisms perceive, this would imply that an organism has an innate set of perceivable affordances, which cannot be infinite in size due to physical limitations of our bodies. Adding a new affordance to this finite set would require first discovering the affordance through perception, perceiving it using more primitive features. Hence, it could be argued that affordances are perceived, and are not the only means through which organisms perceive. For example, humans are good at visual object recognition (i.e. telling the name of an object they see), and this is a useful perceptual task that has little to do with affordances (though some of the representations might be shared between affordance perception and object recognition, as suggested by neuroscience (Krüger et al., 2011; Kruger et al., 2012) and explored in robotics (Dehban et al., 2022). Conversely, nothing precludes an agent from later using affordances as means for perception in order to attribute semantic meaning to objects in his environment (Do et al., 2018).

To resume, we perceive more than just affordances, including objects that offer no affordances (e.g. due to their distance or physical constitution, such as clouds; or abstract concepts, such as beauty). This being said, affordance perception, and more broadly ecological perception, is a central cognitive process of many (if not all) living organisms, from primates (Cisek, 2007) to cockroaches (Sponberg and Full, 2008), and maybe even plants (Frazier et al., 2020). In the primate brain this has given rise to multiple visual processing streams, as evidenced by patients with brain lesions who can recognize but not handle certain objects or, conversely, can handle objects without being able to name them or even without being consciously aware of them (Milner and Goodale, 1995).
## Why is affordance any better than stimulus? What does a theory of affordance suggest that stimuli cannot? How has it moved the needle past Gibson 1960's recognition of how little we can define stimuli?

Stimulus is an entity (object, event, etc.) in the environment that triggers the agent's perceptual processes to generate a response. Although it is used broadly in Psychology, the distinction on whether it refers to the actual entity in the environment, or perceptual processes triggered within the agent usually remains vague. Nevertheless, the concept of a stimulus shares some of the properties of affordance: first, it is minimal, meaning that it refers to the minimal set of perceptions that can elicit a particular response. Second, it is directly linked to action, meaning that perceptions that do not elicit responses from the agent are not considered stimuli. The main difference between a stimulus and an affordance is that a stimulus automatically elicits a response without taking into account the conscious goal of the agent. The stimulus is part of a reactive control loop, and would at least trigger the diversion of attention onto itself. An affordance, on the other hand, is not a perceptual process that is linked with an automatic response. It allows the agent to observe directly (automatically, unconsciously) what the environment offers, but then use this to achieve its conscious goals.

**Affordance versus stimulus.** An affordance can be seen as a tuple combining five elements: the object, agent, action, effect, and environment; whereas a stimulus describes only the object/event that generates a reaction from the agent. Hence, a stimulus is directly linked to action: the action component is automatically triggered (it is reactive) and does not take into account the conscious goal of the agent.

Gibson (1960) essentially points out that the term *stimulus* is heavily overloaded and is used by psychologists to refer to just about any sensory signal controlled by an experimenter, across a wide range of complexity, from delivering an electric pulse to a single sensory neuron to phrases of speech. In analyzing the use of the term he tries to specify what types of stimuli are useful for what types of psychological experiments.

To study mechanisms and roles of affordance, the notion of *sensory signals* seems more useful. Sensory signals are the signals produced by sensory neurons. Perception is the process of making sense of these signals. Our body constantly produces a flood of sensory signals, most of which are irrelevant and are ignored by perceptual processes most of the time. Perception filters and processes sensory information to reconcile an internal model. Part of what perception produces is affordances.

The perception of affordances starts from the sensing of specific stimuli. Not all stimuli that are available: only those that are relevant to the agent, given the sensorimotor capabilities of the specific agent; only the minimal set that the agent has learned to pick, from a stream of several stimuli. Then, the added value of affordances is that these stimuli are automatically translated into actions, or more precisely, action possibilities: the stimulus itself is not really perceived (e.g. represented, stored, memorised, reasoned upon), but instead it is directly converted (i.e. *direct perception*) into an action representation. It should be noted, at this point, that while Gibson was defining affordance perception from a theoretical point of view (he was a theoretical psychologist, after all), those who are interested in using the notion of affordances to improve autonomous systems (e.g. robots) might not find the word *direct* very useful. For example, in the case of robotics, visual stimuli are picked by a camera and translated into an electronic signal and then a matrix of numbers; to generate the action representations, some computation is nevertheless required, whether it is by *direct* perception or by representing the stimuli and then reasoning upon them. However, the useful part of the notion of affordances is that this computation can be *minimal* and *fast*; to obtain these useful characteristics, the computation has to assume the sensorimotor capabilities of the agent as a prior, and this is typically obtained by previous learning (i.e. by exploring the environments using its own sensorimotor abilities, the agent has learned what is the *minimal* amount of information to pick from the visual stream, and therefore the generation of an action representation from such picked information can be *fast* and effective).

**Fast affordance detection.** It is not evident how useful it is to insist that affordance detection be *minimal* or *fast*. No action opportunity pops abruptly into existence; they are part of the agents' environment, of which the agent maintains a continually-evolving internal model (at fluctuating levels of detail, precision, accuracy, and awareness). At what level of computational cost of detection does an affordance cease to be an affordance? Moreover, robustness can be expected to be ecologically more important than (and at odds with) minimality of sensory signals involved.

By the same argument, Gibson's notion of *direct perception* does not seem crucially relevant to affordances. Directness of perception mostly refers to no reliance on memory, inference, or learning. This largely limits the level of sophistication and abstraction of direct perception to shape-from-X, which plausibly includes the detection of some but not all affordances.

Affordance provides several benefits:
* It allows to create semantic maps of the environment, by labelling objects with affordances they provide.

* It allows to identify a missing element within an affordance tuple, based on its experience (in the form of known affordances). For example, it allows to identify an appropriate action on a given object to obtain a desired effect (Awaad et al., 2013).

* Since affordances incorporate an object's physical description, they can be used in object design (Andries et al., 2020). Conceptually, objects could be designed based on a list of affordances they must provide, using knowledge on corresponding physical shapes providing those affordances.

* It allows to define object interaction interfaces in the object space frame, that are generic with respect to the robot model. For instance, an affordance may be available to any robot capable of performing a specific action (described in the object space frame) on the object. The robot can compute the corresponding action (in its own body frame) using inverse kinematics.

Pragmatic work in robotics, compared to Gibson's intentionally vague definitions of affordances, moved on to formal definitions of affordances that allow unambiguous software implementation on the robot (Chavez-Garcia et al., 2017; Krüger et al., 2011).

**What is the connection between affordances, behavioral scale, and intention? How distant or reluctant can a perceiving-acting system be before affordances come into play? Do affordances exert their role only when the system is close or willing enough?**

**Affordances and intentions.** Although there is no need for intentions in order to perceive affordances, intentions are useful for economical perception of the environment. In the presence of intentions, affordances are possibilites that can be acted upon. It can be argued that an agent always has active implicit or explicit intentions/goals.

Because perceiving affordances is *fast* and requires *minimal* computation, it is not a burden for an agent to perceive affordances that might not be used; however, having the action representations ready can then speed up action execution if the intention (i.e. the need for action) comes to be. Similarly, affordances offered by the immediate environment are perceived faster than affordances that are offered in the future after the execution of a series of actions. In addition, the perception of an action possibility could in fact suggest to the agent that a useful action that the agent did not think about (i.e., an action for which there was no intention at all) is available. This can be extremely useful for robotic systems as well. In fact, most formalizations of affordances in robotics rely on representations that include not only the action but also the effects (or in other terms, the goal) of the action (Dehban et al., 2016, 2017; Gonçalves et al., 2014; Krüger et al., 2011; Montesano et al., 2008; Sarathy and Scheutz, 2016; Stramandinoli et al., 2017; Ugur and Piater, 2016). Therefore, the perceived possibilities for actions (and for achieving certain effects) can be used for action planning, leading to problem solving.

If intention can be formulated as a goal state with specific properties, and if affordances afford actions performing transitions between states (with specific preconditions and effects), then we can perform symbolic planning to identify a sequence of applied affordances that brings us to the desired goal state. In this sense, affordance perception can be seen as a semantic layer of perception. Affordances represent the possible actions that were identified by the agent in the environment, and which are available for planning purposes.

When an agent interacts with its environment, it perceptually parses the environment into relevant affordances. The perceived affordances are necessarily filtered by the agent's behavior or intent, as the contrary would require a finite set of types and instances of affordances to exist within the agent's perceived surroundings. For example, when strolling through a forest among shrubs and branches, there is an infinity of grasp affordances almost everywhere, but they are all useless to the strolling agent. It is wasteful and in fact impossible for a finite brain to process all of them all the time. What the agent cares about is affordances for foot placement, obstacles, etc. However, when the agent contemplates climbing a tree, the world is perceived very differently, and reachable grasp affordances become perceptual building blocks.

Ideally, perceiving affordances should be implemented as a rapid and computationally efficient process, enabling an agent to perceive affordances that may be utilized immediately or later on. Having readily available action representations can expedite action execution when the intention or need for action arises. In this context, affordances offered by the immediate environment are perceived more swiftly than those that arise in the future following a series of executed actions. Simply perceiving the possibility of action can alert the agent to a helpful action that had not been previously considered or intended. This aspect holds significant value, particularly for robotic systems.

In robotics, most formalizations of affordances encompass not only the action itself but also its effects. This comprehensive representation enables the perception of potential actions and associated effects, which can be leveraged by robotic
systems to engage in efficient action planning (for example, enabling them to navigate cluttered environments and solve complex tasks effectively without being overwhelmed by the multitude of possible actions). The computationally fast perception of affordances (comprehending actions and their effects) contributes to improving robotic systems' problem-solving capabilities and overall performance.

**How do systems engage with affordances as they move amongst tasks and intentions?**

**Affordances and tasks in cognitive robotics.** When the concept of affordances is used in autonomous systems (e.g., robots), it becomes evident that affordance perception is only a part of a larger cognitive architecture. Robotic approaches that relied on affordances and ecological perception alone did achieve interesting reactive behaviours (Arkin, 1998; Brooks, 1990; Duchon et al., 1998; Murphy, 1999), but could not scale to more complex tasks. However, the integration of affordance perception with other cognitive processes (e.g. object recognition, attention, semantic reasoning, inhibition, action planning) within larger cognitive architectures (Antunes et al., 2016; Franklin et al., 2013; Krüger et al., 2009; Sun, 2007; Vernon et al., 2011) has enabled robots to realize complex problem solving in unstructured environments.

**Observation and inference of affordances.** Affordances offer themselves to the agent; the agent filters them by tasks and intentions, and may perceive them consciously, subconsciously, or not at all. The agent may actively look for them, or even infer them based on indirect evidence (such as observing another climber holding onto something invisible from the current vantage point).

**Planning using affordances.** Since actions are characterized by their effects, affordances can serve as planning operators. In addition to planning over a fixed set of perceived affordances, problem solving may prompt the agent to actively look for specific affordances to form a feasible or improved plan.

The goal of the agent (defined by the tasks and/or intentions) require the agent to plan (either through forward chaining, or other mechanisms) a sequence of actions that would take the agent from the current environment to a desired state of the environment. Such “planning” requires the agent to start with the current environment which is immediately perceivable to pan out and evaluate the possible courses of actions that would bring the agent closer to its goal. Affordances support this process in two ways: First, the affordances offered by the environment (such as objects) can be used to define and limit the action set for planning. Second, the goals (or subgoals) can be used to filter the set of affordances to be sought after in the environment preventing the agent to be “drowning” in affordances.

In this planning context, an action described by an affordance, with its preconditions (object descriptions allowing the action, agent's physical and logical capabilities) and its postconditions (description of effects of the applied action) can be seen as components of a planning domain. Thus, classical planning methods (McDermott et al., 1998) can be used to plan action sequences for executing tasks and reaching intended goals (Ahmetoglu et al., 2022).

**Guiding learning using affordances.** Conversely, affordances offer opportunities for exploration. If the effect of an action is insufficiently known, the agent may just try it out. This may lead to the discovery of new affordances.

**Concluding remarks**

The consensus is that an affordance is an opportunity for action offered by an object to an agent. Formal definitions present it as a relationship between features of an object, an agent, an action, an effect, and an environment.

Affordances are discovered through sensorimotor exploration of the environment. Perceiving affordances allows to attribute semantic meaning to objects in the environment. An argument was provided suggesting affordance perception is not the only type of perception. In the domain of robotics, where multiple types of sensorial processing are commonplace (object detection, segmentation, recognition, etc.), this is definitely true.

The concept of affordance is more complex than that of “stimulus.” In a sense, an affordance provides a direct link between a stimulus and an action possibility.

It is debatable whether affordance perception requires the presence of goals/intentions. On one hand, discovering affordances present in the environment does not require a specific goal (other than the goal of learning how the environment behaves). On the other hand, goals help to filter affordances into useless and useful ones (for reaching a specific goal). Intentionally scanning the environment only for specific affordances allows for economic perception.

Integrating affordance perception with cognitive processes like semantic reasoning and action planning endows robots with the ability to compute action sequences for reaching their intended goals.
Like reinforcement learning and developmental (epigenetic) robotics, the affordance is yet another concept originating in psychology that has found practical applications in robotics. Autonomous cognitive robots being still an immature technology, the evolution of the use of affordances in robotics is worth following closely.

References

Ahmetoglu, A., Seker, M. Y., Piater, J., Oztop, E., and Ugur, E. (2022). DeepSym: Deep symbol generation and rule learning for planning from unsupervised robot interaction. *Journal of Artificial Intelligence Research*, 75:709–745.
https://doi.org/10.1613/jair.1.13754.

Akbulut, M. T. and Ugur, E. (2020). Learning object affordances from sensory-motor interaction via Bayesian networks with auto-encoder features. *International Journal of Intelligent Systems and Applications in Engineering*, 8(2):52–59.
https://doi.org/10.18201/ijisae.2020261584.

Andries, M., Dehban, A., and Santos-Victor, J. (2020). Automatic generation of object shapes with desired affordances using voxelgrid representation. *Frontiers in Neurorobotics*, 14:22. https://doi.org/10.3389/fnbot.2020.00022.

Antunes, A., Jamone, L., Saponaro, G., Bernardino, A., and Ventura, R. (2016). From human instructions to robot actions: Formulation of goals, affordances and probabilistic planning. In *2016 IEEE International Conference on Robotics and Automation (ICRA)*, pages 5449–5454. IEEE. https://doi.org/10.1109/ICRA.2016.7487757.

Arkin, R. C. (1998). *Behavior-based Robotics*. MIT Press, Cambridge, MA.

Awaad, I., Kraetzschmar, G. K., and Hertzberg, J. (2013). Affordance-based reasoning in robot task planning. In *Planning and Robotics (PlanRob) Workshop ICAPS-2013*, page 2.

Brooks, R. A. (1990). Elephants don't play chess. *Robotics and Autonomous Systems*, 6(1-2):3–15. https://doi.org/10.1016/S0921-8890(05)80025-9.

Chavez-Garcia, R. O., Andries, M., Luce-Vayrac, P., and Chatila, R. (2017). Discovering and manipulating affordances. In *2016 International Symposium on Experimental Robotics*, pages 679–691. Springer, Cham, Switzerland. https://doi.org/10.1007/978-3-319-50115-4_59.

Cisek, P. (2007). Cortical mechanisms of action selection: The affordance competition hypothesis. *Philosophical Transactions of the Royal Society B: Biological Sciences*, 362(1485):1585–1599. https://doi.org/10.1098/rstb.2007.2054.

Dehban, A., Jamone, L., Kampff, A. R., and Santos-Victor, J. (2016). Denoising auto-encoders for learning of objects and tools affordances in continuous space. In *2016 IEEE International Conference on Robotics and Automation (ICRA)*, pages 4866–4871. IEEE. https://doi.org/10.1109/ICRA.2016.7487691.

Dehban, A., Jamone, L., Kampff, A. R., and Santos-Victor, J. (2017). A deep probabilistic framework for heterogeneous self-supervised learning of affordances. In *2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids)*, pages 476–483. IEEE. https://doi.org/10.1109/HUMANOIDS.2017.8246915.

Dehban, A., Zhang, S., Cauli, N., Jamone, L., and Santos-Victor, J. (2022). Learning deep features for robotic inference from physical interactions. *IEEE Transactions on Cognitive and Developmental Systems*. https://doi.org/10.1109/TCDS.2022.3152383.

Do, T.-T., Nguyen, A., and Reid, I. (2018). Affordancenet: An end-to-end deep learning approach for object affordance detection. In *2018 IEEE International Conference on Robotics and Automation (ICRA)*, pages 5882–5889. IEEE. https://doi.org/10.1109/ICRA.2018.8460902.

Duchon, A. P., Kaelbling, L. P., and Warren, W. H. (1998). Ecological robotics. *Adaptive Behavior*, 6(3-4):473–507.
https://doi.org/10.1177/105971239800600306.

Franklin, S., Madl, T., D'mello, S., and Snaider, J. (2013). LIDA: A systems-level architecture for cognition, emotion, and learning. *IEEE Transactions on Autonomous Mental Development*, 6(1):19–41. https://doi.org/10.1109/TAMD.2013.2277589.

Frazier, P. A., Jamone, L., Althoefer, K., and Calvo, P. (2020). Plant bioinspired ecological robotics. *Frontiers in Robotics and AI*, 7:79. https://doi.org/10.3389/frobt.2020.00079.

Gibson, E. J. (1994). *An Odyssey in Learning and Perception*. MIT Press, Cambridge, MA.

Gibson, E. J. (2000). Perceptual learning in development: Some basic concepts. *Ecological Psychology*, 12(4):295–302.
https://doi.org/10.1207/S15326969EC01204_04.

Gibson, E. J. (2003). The world is so full of a number of things: On specification and perceptual learning. *Ecological Psychology*, 15(4):283–287. https://doi.org/10.1207/s15326969eco1504_3.
Gibson, J. J. (1960). The concept of the stimulus in psychology. *American Psychologist*, 15(11):694. https://psycnet.apa.org/doi/10.1037/h0047037.

Gibson, J. J. (1966). *The Senses Considered as Perceptual Systems*. Houghton Mifflin, Boston, MA.

Gonçalves, A., Abrantes, J., Saponaro, G., Jamone, L., and Bernardino, A. (2014). Learning intermediate object affordances: Towards the development of a tool concept. In *4th International Conference on Development and Learning and on Epigenetic Robotics*, pages 482–488. IEEE. https://doi.org/10.1109/DEVLRN.2014.6983027.

Krüger, N., Geib, C., Piater, J., Petrick, R., Steedman, M., Wörgötter, F., Ude, A., Asfour, T., Kraft, D., Omrčen, D., et al. (2011). Object-action complexes: Grounded abstractions of sensory-motor processes. *Robotics and Autonomous Systems*, 59(10):740–757. https://doi.org/10.1016/j.robot.2011.05.009.

Kruger, N., Janssen, P., Kalkan, S., Lappe, M., Leonardis, A., Piater, J., Rodriguez-Sanchez, A. J., and Wiskott, L. (2012). Deep hierarchies in the primate visual cortex: What can we learn for computer vision? *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 35(8):1847–1871. https://doi.org/10.1109/TPAMI.2012.272.

Krüger, N., Piater, J., Wörgötter, F., Geib, C., Petrick, R., Steedman, M., Ude, A., Asfour, T., Kraft, D., Omrcen, D., et al. (2009). A formal definition of object-action complexes and examples at different levels of the processing hierarchy. *PACO-PLUS Technical Report*.

McDermott, D., Ghallab, M., Howe, A., Knoblock, C., Ram, A., Veloso, M., Weld, D., and Wilkins, D. (1998). PDDL-the planning domain definition language.

Milner, A. D. and Goodale, M. A. (1995). *The visual brain in action*. Oxford University Press.

Min, H., Luo, R., Zhu, J., Bi, S., et al. (2016). Affordance research in developmental robotics: A survey. *IEEE Transactions on Cognitive and Developmental Systems*, 8(4):237–255. https://doi.org/10.1109/TCDS.2016.2614992.

Montesano, L., Lopes, M., Bernardino, A., and Santos-Victor, J. (2008). Learning object affordances: From sensory-motor coordination to imitation. *IEEE Transactions on Robotics*, 24(1):15–26. https://doi.org/10.1109/TR0.2007.914848.

Murphy, R. R. (1999). Case studies of applying Gibson's ecological approach to mobile robots. *IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans*, 29(1):105–111. https://doi.org/10.1109/3468.736365.

Myers, A., Teo, C. L., Fermüller, C., and Aloimonos, Y. (2015). Affordance detection of tool parts from geometric features. In *2015 IEEE International Conference on Robotics and Automation (ICRA)*, pages 1374–1381. IEEE. https://doi.org/10.1109/ICRA.2015.7139369.

Rosenthal, S., Biswas, J., and Veloso, M. M. (2010). An effective personal mobile robot agent through symbiotic human-robot interaction. In *AAMAS '10: The Ninth International Conference on Autonomous Agents and Multiagent Systems*, volume 10, pages 915–922. ACM, New York, NY.

Sarathy, V. and Scheutz, M. (2016). A logic-based computational framework for inferring cognitive affordances. *IEEE Transactions on Cognitive and Developmental Systems*, 10(1):26–43. https://doi.org/10.1109/TCDS.2016.2615326.

Sponberg, S. and Full, R. (2008). Neuromechanical response of musculo-skeletal structures in cockroaches during rapid running on rough terrain. *Journal of Experimental Biology*, 211(3):433–446. https://doi.org/10.1242/jeb.012385.

Stramandinoli, F., Tikhanoff, V., Pattacini, U., and Nori, F. (2017). Heteroscedastic regression and active learning for modeling affordances in humanoids. *IEEE Transactions on Cognitive and Developmental Systems*, 10(2):455–468. https://doi.org/10.1109/TCDS.2017.2700207.

Sun, R. (2007). The importance of cognitive architectures: An analysis based on CLARION. *Journal of Experimental & Theoretical Artificial Intelligence*, 19(2):159–193. https://doi.org/10.1080/09528130701191560.

Ugur, E. and Piater, J. (2016). Emergent structuring of interdependent affordance learning tasks using intrinsic motivation and empirical feature selection. *IEEE Transactions on Cognitive and Developmental Systems*, 9(4):328–340. https://doi.org/10.1109/TCDS.2016.2581307.

Vernon, D., Von Hofsten, C., and Fadiga, L. (2011). *A Roadmap for Cognitive Development in Humanoid Robots*, volume 11. Springer Science & Business Media, Berlin/Heidelberg, Germany.