A method implemented by one or more processors, comprising: receiving a plurality of observations (140A to N) from a plurality of agents (130A to N) in an environment, each of the observations generated based on sensor data from one or more sensors of a corresponding agent of the agents and including: an observation time, a source identifier of the corresponding agent, an observed object identifier of a corresponding observed object of multiple objects (105A to M, 130A to N) of the environment, a measured object pose for the corresponding observed object, and an observation uncertainty measure for the measured object pose; identifying a group of the observations for a target time based on the observation times of the observations of the group satisfying a threshold relative to the target time; generating, based on the group of observations, a multigraph having a plurality of nodes (1 to 5) and edges connecting the nodes, the generating comprising defining a reference frame node of the nodes, defining object nodes of the nodes that are based on the observed object identifiers of the group, and generating the edges connecting the nodes based on the observed object identifiers, the source identifiers, and the measured object poses and observation uncertainty measures of the group; generating a composite pose and composite uncertainty measure for each of a plurality of simple paths along the edges that connect the reference frame node and a given object node of the object nodes, wherein the composite pose and the composite uncertainty measure for each of the simple paths is based on the measured object pose and the observation uncertainty measure for each of the edges of the simple path; generating, based on the composite poses and the composite uncertainty measures, a pose and an uncertainty measure for the object identifier on which the given object node is based; and assigning the pose and the uncertainty measure to the object identifier on which the given object node is based and/or providing the object identifier and the pose and the uncertainty measure to at least one robotic agent of the agents in the environment to influence one or more tasks to be performed by the robotic agent. receiving a plurality of observations (140A to N) from a plurality of agents (130A to N) in an environment, each of the observations generated based on sensor data from one or more sensors of a corresponding agent of the agents and including: an observation time, a source identifier of the corresponding agent, an observed object identifier of a corresponding observed object of multiple objects (105A to M, 130A to N) of the environment, a measured object pose for the corresponding observed object, and an observation uncertainty measure for the measured object pose; identifying a group of the observations for a target time based on the observation times of the observations of the group satisfying a threshold relative to the target time; generating, based on the group of observations, a multigraph having a plurality of nodes (1 to 5) and edges connecting the nodes, the generating comprising defining a reference frame node of the nodes, defining object nodes of the nodes that are based on the observed object identifiers of the group, and generating the edges connecting the nodes based on the observed object identifiers, the source identifiers, and the measured object poses and observation uncertainty measures of the group; generating a composite pose and composite uncertainty measure for each of a plurality of simple paths along the edges that connect the reference frame node and a given object node of the object nodes, wherein the composite pose and the composite uncertainty measure for each of the simple paths is based on the measured object pose and the observation uncertainty measure for each of the edges of the simple path; generating, based on the composite poses and the composite uncertainty measures, a pose and an uncertainty measure for the object identifier on which the given object node is based; and assigning the pose and the uncertainty measure to the object identifier on which the given object node is based and/or providing the object identifier and the pose and the uncertainty measure to at least one robotic agent of the agents in the environment to influence one or more tasks to be performed by the robotic agent.