A method of performing autonomous perception by an autonomous perception system, the method comprising: acquiring perception sensor data as an image by the autonomous perception system, the autonomous perception system comprising a processing system coupled to a perception sensor system; performing feature extraction on the image by the autonomous perception system, the feature extraction identifying one or more features in the image; determining contextual information associated with one or more conditions present upon acquiring the perception sensor data, the contextual information including one or more of a time of day, a season of the year, and weather conditions; retrieving one or more semantically labeled reference images from a database based on the contextual information and at least one or more features extracted from the image, wherein the database is implemented as a contextually-indexed database based on contextual information and wherein the retrieving one or more semantically labeled reference images includes identifying the one or more semantically labeled reference images from the database based on the contextual information and having a highest level of similarity with at least one of the one or more features extracted, wherein the highest level of similarity is determined based on comparing the one or more features extracted to a plurality of labeled reference images in the database having a contextual index value matching the contextual information and ranking results of the comparing; and parsing the image into segments for labeling and transferring one or more semantic labels from the one or more semantically labeled reference images to the image to form a semantically labeled version of the image. acquiring perception sensor data as an image by the autonomous perception system, the autonomous perception system comprising a processing system coupled to a perception sensor system; performing feature extraction on the image by the autonomous perception system, the feature extraction identifying one or more features in the image; determining contextual information associated with one or more conditions present upon acquiring the perception sensor data, the contextual information including one or more of a time of day, a season of the year, and weather conditions; retrieving one or more semantically labeled reference images from a database based on the contextual information and at least one or more features extracted from the image, wherein the database is implemented as a contextually-indexed database based on contextual information and wherein the retrieving one or more semantically labeled reference images includes identifying the one or more semantically labeled reference images from the database based on the contextual information and having a highest level of similarity with at least one of the one or more features extracted, wherein the highest level of similarity is determined based on comparing the one or more features extracted to a plurality of labeled reference images in the database having a contextual index value matching the contextual information and ranking results of the comparing; and parsing the image into segments for labeling and transferring one or more semantic labels from the one or more semantically labeled reference images to the image to form a semantically labeled version of the image.