Patent ID: 12245810

DESCRIPTION OF EXAMPLE EMBODIMENTS

General Overview

In one embodiment, a method for validating a vision system includes providing a vision testing medium at a first distance from the vision system, communicating a first information relating to the vision testing medium from the sensing apparatus to a first system, and displaying, using the first system, a rendering of the vision testing medium based on the first information. The method also includes obtaining, using the first system, at least a first indication of a visual acuity associated with the vision system, determining when the visual acuity at least meets a threshold level, and identifying the sensing apparatus as validated when the visual acuity at least meets the threshold level.

In another embodiment, logic is encoded in one or more tangible non-transitory, computer-readable media for execution. When executed, the logic is operable to provide a vision testing medium at a first distance from the vision system, to communicate a first information relating to the vision testing medium from the sensing apparatus to a first system, and to display, using the first system, a rendering of the vision testing medium based on the first information. The logic is further operable to obtain, using the first system, at least a first indication of a visual acuity associated with the vision system, to determine when the visual acuity at least meets a threshold level; and to identify the sensing apparatus as validated when the visual acuity at least meets the threshold level.

In yet another embodiment, a method includes obtaining a first information from a simulation system, the simulation system configured to simulate a driving environment, the first information including a first indication associated with a vision testing medium rendered using the simulation system, the first information being arranged to indicate a visual acuity associated with the simulation system. The method also includes comparing the first information to a visual acuity associated with a vision system of a vehicle. Comparing the first information to the visual acuity associated with the vision system of the vehicle includes determining whether the first information indicates at least an approximate match between the visual acuity associated with the simulation system and the visual acuity associated with the vision system of the vehicle. Finally, the method includes adjusting at least one parameter associated with the simulation system when it is determined that the first information does not indicate the at least approximate match between the visual acuity associated with the simulation system and the visual acuity associated with the vision system of the vehicle, and identifying the simulation system as calibrated when it is determined that the first information indicates the at least approximate match between the visual acuity associated with the simulation system and the visual acuity associated with the vision system of the vehicle.

In still another embodiment, a vision system associated with a teleoperation system is effectively validated through the use of at least one vision acuity test. A vision system, which may include cameras and/or other sensors arranged to provide a visual depiction of an environment, may be used provide a photograph or a video of a vision testing medium to a human via a teleoperations system. The vision testing medium may be an eye chart, e.g., a Landolt C eye chart. A human may view the vision testing medium using the teleoperations system. Different components of the vision system may be tested by placing the eye chart at different locations relative to the vision system, as for example such that the human may effectively see what the vision system sees with respect to different sides of the vehicle. The performance of the human when reading or otherwise deciphering the vision testing medium may be assessed to determine whether the vision system meets standards. When the performance meets standards set for human drivers, as for example when human drivers are tested when applying for or renewing drivers licenses, then a vision system and, hence, a teleoperations system may be considered to be validated.

DESCRIPTION

As the use of autonomous vehicles grows, the ability for the autonomous vehicles to operate safely is becoming increasingly important. Teleoperation systems are often used to facilitate the safe operation of autonomous vehicles. A teleoperation system may be used to remotely operate an autonomous vehicle when the autonomous vehicle in need of assistance, as for example when the autonomous vehicle is unable to safely navigate a particular roadway or otherwise respond to an unexpected situation. Other systems such as simulation systems are used to increase the likelihood that when autonomous vehicles are operating on roads, the autonomous vehicles are operating safely.

In order for a teleoperation system to take control of a vehicle when an autonomy system of the vehicle effectively relinquishes control to the teleoperation system, video or other vision data provided by the vehicle to the teleoperation system is used by the teleoperation system to substantially navigate the vehicle. If the video or vision data is not of an adequate resolution or quality to enable a teleoperator, or a remote human driver, to safely operate the vehicle, then the vehicle may pose a danger in its environment. Thus, it is important to essentially ensure that the video or vision data provided to a teleoperation system by a vehicle, or by a vision system of the vehicle, meets acceptable standards.

In one embodiment, the vision system of an autonomous vehicle may effectively be calibrated using one or more vision testing mediums that are generally used to test human vision. A vision testing medium may be an eye chart such as a Landolt C eye chart, or a Landolt eye chart, that is placed at a predetermined distance from a vision system of a vehicle, and a visual image of the vision testing medium may be transmitted to a teleoperation system. A human may view the visual image using the vision system of the vehicle, and if the human is able to accurately discern the vision testing medium in the visual image, then the vision system may be determined to meet acceptable standards. In other words, a human who may remotely operate a vehicle using a teleoperation system may effectively be tested in the same way that he or she would be tested to be licensed to drive a vehicle from a cockpit or a driver's seat.

By essentially administering an eye test to a human using a vision system of an autonomous vehicle in cooperation with a teleoperations system, the vision system may be substantially validated. For example, a vision testing medium may be positioned with respect to a vision system that is such that a human who is viewing the vision testing medium through a teleoperation system is effectively viewing the vision testing medium from a distance of approximately twenty feet. If the human has vision that meets predetermined standards for a static visual acuity as determined using the vision testing medium, then the vision system may be considered to be validated. While some standards may specify that a human needs to test as having substantially perfect vision in terms of clearness and sharpness of vision, e.g., approximately 20/20 vision, in order to safely operate a vehicle, other standards may specify that a human may be deemed able to safely operate a vehicle with less than perfect vision, e.g., approximately 20/40 vision. In general, if a human is able to clearly and sharply discern the vision testing medium transmitted to a display screen by a vision system, then the vision system may be considered to be validated.

In one embodiment, a vision testing medium may be a Landolt eye test chart or a Landolt C eye chart, although it should be appreciated that the vision testing medium is not limited to being a Landolt eye chart or a Landolt C eye chart, A Landolt eye chart includes at least one Landolt C. That is, a Landolt eye chart includes optotypes that are Landolt C's. Landolt C is typically a circle or a ring that has an opening or a gap within the ring such that the Landolt C approximates the letter “C.” A Landolt C, which may also be known as a Landolt ring, a Landolt broken ring, or a broken ring, may be rotated such that the gap may be in different positions with respect to the ring that includes the gap. For example, the gap may be located at approximately zero degrees or in an “up” position, approximately 90 degrees or in a “right position,” approximately 180 degrees or in a “down position,” and/or approximately 270 degrees or in a “left position” along the ring. Generally, a substantially minimum perceivable angle of the gap may be considered to be a measurement of visual acuity,

A Landolt eye chart may be positioned at a distance away from a vision system that, when calibrated with respect to a distance between a human and a teleoperations system, provides an effective distance of approximately twenty feet between the Landolt C eye chart and the human. In one embodiment, a Landolt eye chart may be positioned at a distance of approximately twenty feet away from a vision system of a vehicle, and an individual viewing the Landolt eye chart using a display screen of a teleoperations system may be considered to be viewing the Landolt eye chart from approximately twenty feet away. Alternatively, a distance between an individual and a display screen of a teleoperations system may be accounted for when determining how many feet away from a vision system of a vehicle a Landolt eye chart is to be positioned in order for the individual to effectively be considered to be viewing the Landolt eye chart from a distance of approximately twenty feet.

Visual acuity, in one embodiment, is a measure of a spatial resolution of an optical system. For example, visual acuity may be defined for the human eye as a function of a gap size, or a testing distance. A visual acuity “score” for a human may measure an ability of the human to resolve detail in a scene, image, or object. For example, the smallest optotype within which an individual may identify or otherwise recognize a critical detail may substantially determine a visual acuity score of the individual. A visual acuity score for an individual, as defined with respect to a Landolt eye test, may be based upon the smallest “C” for which the individual ay identify a location of a gap, which may be considered to be a critical detail.

An autonomous vehicle that may be operated using a teleoperation system may generally be part of a fleet of autonomous vehicles. Referring initially toFIG.1, an autonomous vehicle fleet will be described in accordance with an embodiment. An autonomous vehicle fleet100includes a plurality of autonomous vehicles101, or robot vehicles. Autonomous vehicles101are generally arranged to transport and/or to deliver cargo, items, and/or goods. Autonomous vehicles101may be fully autonomous and/or semi-autonomous vehicles. In general, each autonomous vehicle101may be a vehicle that is capable of travelling in a controlled manner for a period of time without intervention, e.g., without human intervention. As will be discussed in more detail below, each autonomous vehicle101may include a power system, a propulsion or conveyance system, a navigation module, a control system or controller, a communications system, a processor, and a sensor system.

Dispatching of autonomous vehicles101in autonomous vehicle fleet100may be coordinated by a fleet management module (not shown). The fleet management module may dispatch autonomous vehicles101for purposes of transporting, delivering, and/or retrieving goods or services in an unstructured open environment or a closed environment.

FIG.2is a diagrammatic representation of a side of an autonomous vehicle, e.g., one of autonomous vehicles101ofFIG.1, in accordance with an embodiment. Autonomous vehicle101, as shown, is a vehicle configured for land travel. Typically, autonomous vehicle101includes physical vehicle components such as a body or a chassis, as well as conveyance mechanisms, e.g., wheels. In one embodiment, autonomous vehicle101may be relatively narrow, e.g., approximately two to approximately five feet wide, and may have a relatively low mass and relatively low center of gravity for stability. Autonomous vehicle101may be arranged to have a working speed or velocity range of between approximately one and approximately forty-five miles per hour (mph), e.g., approximately twenty-five miles per hour. In some embodiments, autonomous vehicle101may have a substantially maximum speed or velocity in range between approximately thirty and approximately ninety mph.

Autonomous vehicle101includes a plurality of compartments102. Compartments102may be assigned to one or more entities, such as one or more customer, retailers, and/or vendors. Compartments102are generally arranged to contain cargo, items, and/or goods. Typically, compartments102may be secure compartments. It should be appreciated that the number of compartments102may vary. That is, although two compartments102are shown, autonomous vehicle101is not limited to including two compartments102.

FIG.3is a block diagram representation of an autonomous vehicle, e.g., autonomous vehicle101ofFIG.1, in accordance with an embodiment. An autonomous vehicle101includes a processor304, a propulsion system308, a navigation system312, a sensor system324, a power system332, a control system336, and a communications system340. It should be appreciated that processor304, propulsion system308, navigation system312, sensor system324, power system332, and communications system340are all coupled to a chassis or body of autonomous vehicle101.

Processor304is arranged to send instructions to and to receive instructions from or for various components such as propulsion system308, navigation system312, sensor system324, power system332, and control system336. Propulsion system308, or a conveyance system, is arranged to cause autonomous vehicle101to move, e.g., drive. For example, when autonomous vehicle101is configured with a multi-wheeled automotive configuration as well as steering, braking systems and an engine, propulsion system308may be arranged to cause the engine, wheels, steering, and braking systems to cooperate to drive. In general, propulsion system308may be configured as a drive system with a propulsion engine, wheels, treads, wings, rotors, blowers, rockets, propellers, brakes, etc. The propulsion engine may be a gas engine, a turbine engine, an electric motor, and/or a hybrid gas and electric engine.

Navigation system312may control propulsion system308to navigate autonomous vehicle101through paths and/or within unstructured open or closed environments. Navigation system312may include at least one of digital maps, street view photographs, and a global positioning system (GPS) point. Maps, for example, may be utilized in cooperation with sensors included in sensor system324to allow navigation system312to cause autonomous vehicle101to navigate through an environment.

Sensor system324includes any sensors, as for example LiDAR, radar, ultrasonic sensors, microphones, altimeters, and/or cameras. Sensor system324generally includes onboard sensors which allow autonomous vehicle101to safely navigate, and to ascertain when there are objects near autonomous vehicle101. In one embodiment, sensor system324may include propulsion system sensors that monitor drive mechanism performance, drive train performance, and/or power system levels. Sensor system324includes a vision system326that may include sensors such as cameras and/or other sensors which may effectively obtain visual information. Vision system324may provide information to remote systems (not shown) such as a teleoperation system via communications system340. A teleoperation system (not shown) may be used to remotely operate vehicle101remotely by providing input, via communications system340, to systems of vehicle101such as propulsion system308and navigation system312. In one embodiment, an assessment of whether vision system326may provide a relatively accurate view of the environment around vehicle101to a teleoperation system (not shown) may be made using a visual acuity test such as a Landolt eye test. Such a visual acuity test may be used to test different components or aspects of vision system326, e.g., the visual acuity test and vision system326may be moved relative to each other such that different components or aspects may be substantially evaluated.

Power system332is arranged to provide power to autonomous vehicle101. Power may be provided as electrical power, gas power, or any other suitable power, e.g., solar power or battery power. In one embodiment, power system332may include a main power source, and an auxiliary power source that may serve to power various components of autonomous vehicle101and/or to generally provide power to autonomous vehicle101when the main power source does not have the capacity to provide sufficient power.

Communications system340allows autonomous vehicle101to communicate, as for example, wirelessly, with a fleet management system (not shown) that allows autonomous vehicle101to be controlled remotely. Communications system340generally obtains or receives data, stores the data, and transmits or provides the data to a fleet management system and/or to autonomous vehicles101within a fleet100. The data may include, but is not limited to including, information relating to scheduled requests or orders, information relating to on-demand requests or orders, and/or information relating to a need for autonomous vehicle101to reposition itself, e.g., in response to an anticipated demand.

In some embodiments, control system336may cooperate with processor304to determine where autonomous vehicle101may safely travel, and to determine the presence of objects in a vicinity around autonomous vehicle101based on data, e.g., results, from sensor system324. In other words, control system336may cooperate with processor304to effectively determine what autonomous vehicle101may do within its immediate surroundings. Control system336in cooperation with processor304may essentially control power system332and navigation system312as part of driving or conveying autonomous vehicle101. Additionally, control system336may cooperate with processor304and communications system340to provide data to or obtain data from other autonomous vehicles101, a management server, a global positioning server (GPS), a personal computer, a teleoperations system, a smartphone, or any computing device via the communication module340. In general, control system336may cooperate at least with processor304, propulsion system308, navigation system312, sensor system324, and power system332to allow vehicle101to operate autonomously. That is, autonomous vehicle101is able to operate autonomously through the use of an autonomy system that effectively includes, at least in part, functionality provided by propulsion system308, navigation system312, sensor system324, power system332, and control system336.

As will be appreciated by those skilled in the art, when autonomous vehicle101operates autonomously, vehicle101may generally operate, e.g., drive, under the control of an autonomy system. That is, when autonomous vehicle101is in an autonomous mode, autonomous vehicle101is able to generally operate without a driver or a remote operator controlling autonomous vehicle. In one embodiment, autonomous vehicle101may operate in a semi-autonomous mode or a fully autonomous mode. When autonomous vehicle101operates in a semi-autonomous mode, autonomous vehicle101may operate autonomously at times and may operate under the control of a driver or a remote operator at other times. When autonomous vehicle101operates in a fully autonomous mode, autonomous vehicle101typically operates substantially only under the control of an autonomy system. The ability of an autonomous system to collect information and extract relevant knowledge from the environment provides autonomous vehicle101with perception capabilities. For example, data or information obtained from sensor system324may be processed by a perception system such that the environment around autonomous vehicle101may effectively be perceived.

In order to ensure that vehicle101may be controlled by a teleoperator, or a remote driver using a teleoperation system, vision system326may be tested or calibrated to substantially ensure that vision system326is capable of providing clear and sharp images. A teleoperation system may be used, in one embodiment, to facilitate the validation of the visual acuity of vision system326.

With reference toFIG.4, an overall system which allows for the visual acuity of a vision system to be validated will be described in accordance with an embodiment. An overall system442that facilitates the validation of a vision system on an autonomous vehicle includes autonomous vehicle101and a teleoperation system or equipment446which communicate over a network448. Network448may be, but is not limited to being, a wireless network such as a cellular network, e.g., an LTE network and/or a 3G/4G/5G network, a Wi-Fi network, and/or a Bluetooth network.

Teleoperation system446is configured to be used to remotely operate or otherwise control autonomous vehicle101, as for example when vehicle101is unable to safely operate in an autonomous mode. Teleoperation system446generally includes equipment such as an operator station that includes ate least a steering wheel, acceleration and brake pedals, and a gear shifter. That is, teleoperation system446includes equipment which enables an individual to remotely drive autonomous vehicle101. Teleoperation system446also includes a communications system446a, a display system446b, and a processing system466c. Communications system446ais arranged to enable teleoperation system446to communicate with autonomous vehicle101through network448. Display system446bis arranged to display data, e.g., visual information received from or otherwise obtained from vision system326of vehicle101. Processing system446cis arranged to process signals generated by teleoperation system446such as signals from an operator station, and to translate the signals into control instructions for substantially controlling autonomous vehicle101. Processing system446cmay cooperate with communications system446ato provide teleoperation capabilities.

Vehicle101communicates with teleoperation system446through communications system340. Data, i.e., image data or visual data, obtained through sensor system324may be provided through communications system340and network448to communications system446a. For example, a signal which contains information from vision system326may be provided for display on display system446b. Once information such as images from vision system326are presented to a human who view display system446b, the human may determine whether he or she is able to see the images clearly and sharply.

FIG.5is a diagrammatic representation of a system which facilitates the validation of a vision system of a vehicle using an eye chart and a human in accordance with an embodiment. A vision test medium552, which may be an eye test chart such as a Landolt test chart, may be positioned within range of a vision system326′ of autonomous vehicle101. Vision system326′ may provide an image of vision test medium552to display system446bof teleoperations system446, and an individual556may view the image to determine how clearly and sharply he or she is able to see the image, or at least the portion of the image which depicts vision test medium552.

The actual distance between vision system326′ and vision test medium552may be determined using a variety of suitable criteria. Suitable criteria may include, but are not limited to including, an effective distance from which individual556is to be from vision test medium552, an actual size of vision test medium552, the quality of display system446bof teleoperations system446, etc. In one embodiment, when vision test medium552is a Landolt eye test chart, vision test medium552may be positioned at a specified distance in front of vision system326′, with the specified distance being a function of the sizes of each Landolt C included in vision test medium552. The specified distance may be a distance of up to approximately twenty feet in some embodiments, although it should be understood that the distance may vary widely based on the sizes of each Landolt C in vision test medium552.

It should be appreciated that when vision test medium552is positioned at a predetermined distance in front of vision system326′, visual acuity may be measured in terms of 20/20 vision, 20/40 vision, etc., and it may be determined that individual556, when viewing an image of vision test medium552on display system446, is effectively viewing the image from a distance at which a person with 20/20 vision may see the image clearly. For example, vision test medium552may be positioned twenty feet in front of vision system326′ such that if individual556is able to view an image of vision test medium552clearly, visual acuity may be substantially determined to be 20/20. In another embodiment, vision test medium552may be positioned at a distance from vision system326′ such that, together with a distance between individual556and teleoperation system446, an overall effective viewing distance between individual556and vision test medium552may be a predetermined distance that correlates to a distance from which a person with 20/20 vision may see the image clearly.

When individual556views an image of vision test medium552using display system446b, individual556is effectively determining the visual acuity of vision system326′. If individual556has substantially perfect vision, e.g., approximately 20/20 vision, then individual556may generally be able to accurately identify critical features such as a gap in each Landolt C displayed on vision test medium552that he or she is expected to be able to discern, as for example from an effective distance of approximately twenty feet. As such, visual acuity of vision system,326′ may effectively be validated when individual556may accurately identify critical features. In other words, if individual556is able to clearly and sharply discern vision test medium552, e.g., identify critical features in each Landolt C associated with vision test medium552, when viewed on display system446b, then vision system326′ may effectively be validated.

As mentioned above, a vision system such as vision system326ofFIG.3may include various components. For example, vision system326may include a plurality of sensors including, but not limited to including, cameras, lidars, and/or radars. Sensors within vision system326may generally be located substantially anywhere on or within a vehicle such as vehicle101ofFIGS.2and3. Although vision system326′ is shown as being located substantially atop vehicle101, it should be appreciated that in general, components of vision system326′ are not limited to being located substantially atop vehicle101, and may be located in a variety of different positions on or within vehicle101.

In one embodiment, a vision system may include sensors which are oriented to substantially face different directions. By way of example, a vision system may include cameras or other vision sensors which face forward, face to the right, face backward, and face to the left of a vehicle. In order to determine the visual acuity of such cameras or other vision sensors, a vision test may be substantially administered using each of the cameras or other vision sensors. A vehicle and/or a visual acuity test such as a Landolt eye chart may be arranged to be moved such that different sensors may be evaluated by the visual acuity test. That is, the visual acuity of sensors on different sides with respect to a vision system may be evaluated by moving the vehicle relative to a Landolt eye chart, by moving the Landolt eye chart relative to the vehicle, or both.

With reference toFIGS.6A-D, the performance of visual acuity tests for different sensors included in a vision system will be described in accordance with an embodiment.FIG.6Ais a diagrammatic representation of a vehicle in a first orientation with respect to an eye chart in accordance with an embodiment. Vehicle101includes a vision system326″ which includes a plurality of cameras and/or other vision sensors. While vision system326″ may be located at a substantially single area of vehicle101, as shown, it should be appreciated that vision system326″ may instead be distributed with respect to vehicle101, e.g., vision system326″ may include different components which are substantially spread out with respect to vehicle101.

Visual acuity test552, which may be a Landolt eye test, is arranged at a distance X1from a substantially front edge associated with vision system326″. When vehicle101is oriented with a front edge associated with vision system326″ that essentially faces visual acuity test552, an image of visual acuity test552that is captured by vision system326″ may effectively be used to determine a visual acuity associated with a sensor of vision system326″ that is generally front-facing.

Distance X1may be, in one embodiment, a distance from visual acuity test552to a front edge associated vision system326″ that effectively enables an individual (not shown) who is viewing visual acuity test552through a display screen of a teleoperations system (not shown) to see visual acuity test552as if he or she is viewing visual acuity test552from a predetermined distance associated with substantially perfect visual acuity, e.g., from approximately twenty feet away. Distance X1may, however, vary depending upon factors including, but not limited to including, a size of optotypes on visual acuity test552, a distance between an individual and a display screen, and/or a quality of the display screen.

In order to evaluate the visual acuity associated with a first side of vehicle101, e.g., a right side of vehicle101, either vehicle101or visual acuity test552may be moved such that a first side of vehicle101may substantially face visual acuity test552.FIG.6Bis a representation of vehicle101in a second orientation in which a first side of vehicle101substantially faces visual acuity test552in accordance with an embodiment. Visual acuity test552is arranged at a distance X2from a substantially first or right edge associated with vision system326″. When vehicle101is oriented with a first edge associated with vision system326″ that essentially faces visual acuity test552, an image of visual acuity test552that is captured by vision system326″ may effectively be used to determine a visual acuity associated with a sensor of vision system326″ that is generally side-facing.

Distance X2may be, in one embodiment, a distance from visual acuity test552to a first edge associated vision system326″ that effectively enables an individual (not shown) who is viewing visual acuity test552through a display screen of a teleoperations system (not shown) to see visual acuity test552as if he or she is viewing visual acuity test552from a predetermined distance associated with substantially perfect visual acuity, e.g., from approximately twenty feet away. Distance X2may, however, vary depending upon factors including, but not limited to including, a size of optotypes on visual acuity test552, a distance between an individual and a display screen, and/or a quality of the display screen.

FIG.6Cshows vehicle101in a third orientation in which a back or rear side of vehicle101substantially faces visual acuity test552in accordance with an embodiment. Visual acuity test552is arranged at a distance X3from a substantially back or rear edge associated with vision system326″. When vehicle101is oriented with a back edge associated with vision system326″ that essentially faces visual acuity test552, an image of visual acuity test552that is captured by vision system326″ may effectively be used to determine a visual acuity associated with a sensor of vision system326″ that is generally back-facing.

Distance X3may be, in one embodiment, a distance from visual acuity test552to a back edge associated vision system326″ that effectively enables an individual (not shown) who is viewing visual acuity test552through a display screen of a teleoperations system (not shown) to see visual acuity test552as if he or she is viewing visual acuity test552from a predetermined distance associated with substantially perfect visual acuity, e.g., from approximately twenty feet away. Distance X3may, however, vary depending upon factors including, but not limited to including, a size of optotypes on visual acuity test552, a distance between an individual and a display screen, and/or a quality of the display screen.

FIG.6Dis a representation of vehicle101in a fourth orientation in which a second or left side of vehicle101substantially faces visual acuity test552in accordance with an embodiment. Visual acuity test552is arranged at a distance X4from a substantially second or left edge associated with vision system326″. When vehicle101is oriented with a second edge associated with vision system326″ that essentially faces visual acuity test552, an image of visual acuity test552that is captured by vision system326″ may effectively be used to determine a visual acuity associated with a sensor of vision system326″ that is generally side-facing.

Distance X4may be, in one embodiment, a distance from visual acuity test552to a first edge associated vision system326″ that effectively enables an individual (not shown) who is viewing visual acuity test552through a display screen of a teleoperations system (not shown) to see visual acuity test552as if he or she is viewing visual acuity test552from a predetermined distance associated with substantially perfect visual acuity, e.g., from approximately twenty feet away. Distance X4may, however, vary depending upon factors including, but not limited to including, a size of optotypes on visual acuity test552, a distance between an individual and a display screen, and/or a quality of the display screen.

It should be appreciated that distances X1, X2, X3, X4may be approximately the same. In one embodiment, distances X1, X2, X3, X4may each be approximately twenty feet. In another embodiment, distances X1, X2, X3, X4may each be a particular distance which, when substantially added to a distance between a human and a screen on which visual acuity test552is displayed after being communicated from vehicle101to a display system, e.g., a display screen of a teleoperations system, is approximately twenty feet.

The size of each Landolt C of a Landolt eye test chart may effectively be determined based on a distance at which a gap width in a Landolt C effectively subtends one minute of arc, or a distance at which an external diameter of a ring that forms a Landolt C effectively subtends five minutes of arc.FIG.7is a diagrammatic representation of a Landolt C or ring in accordance with an embodiment. A Landolt C or ring754generally has a circular perimeter, and includes a gap756in the perimeter. That is, Landolt C754is generally a ring with a break therein that is formed by gap756. As shown, gap756is positioned at approximately 90 degrees along the perimeter or arc of Landolt C754. However, it should be appreciated that gap756may generally be positioned substantially anywhere along the perimeter of Landolt C754, as for example at approximately 180 degrees along the perimeter, at approximately 270 degrees along the perimeter, and at approximately zero degrees along the perimeter.

The dimensions of gap756may generally vary. For example, dimensions of gap756relative to an x-axis and/or a y-axis may vary. In one embodiment, a gap dimension such as a width may be a function of a width associated with Landolt C754, That is, a gap size or dimension may vary with the size or dimension of Landolt C754. For example, gap756may measure approximately one minute of the arc associated with Landolt C754. A thickness of the perimeter of Landolt C754may be approximately one-fifth of a diameter (D) of Landolt C754, and a width of gap756may also be approximately one-fifth of the diameter of Landolt C754.

Referring next toFIGS.8A and8B, a method of validating the visual acuity of a vision system will be described in accordance with an embodiment. A method805of validating the visual acuity of a vision system of a vehicle begins at a step809in which a first distance between the vision system and a vision testing medium is determined when the vehicle is in a position N, e.g., a first position which may be a substantially front-facing position. Determining such a first distance may generally include, but is not limited to including, determining a distance that is consistent with a desired vision acuity validation distance, as will be discussed below. In one embodiment, the first distance may be a linear distance between a vision system mounted on an autonomous vehicle and an eye test chart such as a Landolt eye chart.

In an optional step813, a second distance may be determined. The second distance is generally a distance between an individual and a display screen, e.g., a display screen of a teleoperations system, that is to be used in a validation process. The first distance and the second distance, together, may substantially define a desired vision acuity validation distance. In one embodiment, a desired vision acuity validation distance may be a function of the first distance and the second distance. For example, a first distance and a second distance may be selected such that a desired vision acuity validation distance, or an effective distance at which a human views a vision testing medium, is equivalent to approximately twenty feet. It should be appreciated that in some embodiments, a first distance may be approximately equal to the desired vision acuity validation distance. When the first distance is approximately equal to the desired vision acuity validation distance, optional step813may be substantially skipped.

From step809or an optional step813, process flow proceeds to an optional step817in which it is verified whether an individual who is to participate in the validation process has acceptable vision or otherwise meets or exceeds minimum vision standards. Acceptable vision may be, for example, vision that is considered to be substantially adequate in view of standards or laws which specify a minimum visual acuity that is to be met by someone licensed to operate a motor vehicle. Acceptable vision may be defined to be associated with a level of visual acuity that meets or exceeds a particular threshold level. While acceptable vision may be defined in some jurisdictions as at least 20/40 or better in at least one eye, it should be appreciated that definitions of acceptable vision may vary widely. Verifying that the human has acceptable vision may include, but is not limited to including, subjecting the human to an eye test which may be a Landolt C test or any other suitable step. Step817is generally optional, as an individual with a valid driver's license will typically have met vision standards for operating a motor vehicle.

In a step821, the vision system gathers visual or image data, and transmits the visual data to a display system which may be viewed by an individual. Transmitting the visual data may include providing or sending the visual data through a network to the display system. The network may be any suitable wireless and/or wireless network. For example, the network may be a 3G/4G/5G network, an LTE network, and/or a Wi-Fi network.

Once the display system obtains the visual data, the visual data is rendered and displayed on the display system in a step823. Displaying the visual data generally includes displaying a rendering of the vision testing medium. In one embodiment, displaying the visual data includes displaying a captured image of a Landolt eye chart on a display system of a teleoperations system.

Information relating to the displayed visual data is collected from an individual in a step825. That is, information pertaining to how the individual perceives the displayed visual data is obtained. It should be appreciated that any suitable method may be used to collected information from an individual. By way of example, the individual may be asked to use a user interface to enter what he or she sees in the visual data into a computer application running on a computing system, or the individual may speak what he or she sees into a recording system or to another human who notes what the individual speaks.

After information is collected from an individual, it is determined in a step827whether the vehicle is to be rotated or otherwise moved relative to the vision testing medium. In other words, if is determined if the vision system of the vehicle is to be further evaluated, e.g., whether other sensors of the vision system such as sensors that face a different direction are to be further evaluated.

If the determination in step827is that the vehicle is to be rotated or otherwise moved relative to the vision testing medium, position N is incremented and the vehicle is rotated, and process flow returns to step809in which a first distance between a vision system and a vision testing medium are determined with the vehicle in position N. On the other hand, if the determination is that the vehicle is not to be rotated or otherwise moved, the implication is that sufficient information has been collected, e.g., substantially every sensor associated with vision system has been evaluated. Accordingly, process flow moves from step827to a step829in which visual acuity is assessed using information collected from the individual. One method of assessing the visual acuity of the human using collected information will be discussed below with respect toFIG.9.

From step829, process flow proceeds to a step833in which it is determined whether the visual acuity is acceptable. Such a determination may include determining whether the visual acuity of the individual, as assessed in step829, indicates that at least minimum standards have been met and/or that a threshold level has been met or exceeded. If the determination is that the visual acuity is not acceptable, then the vision system is identified as not meeting standards, and is not considered to be validated in a step837. The method of validating the visual acuity of a vision system is terminated upon identifying the vision system as not validated.

Alternatively, if the determination in step833is that the visual acuity is acceptable, the implication is that individual has met at least minimum standards through the use of the vision system. As such, the vision system may be considered to be capable of providing images to a teleoperator that have at least an adequate level of clearness and sharpness. Accordingly, process flow moves from step833to a step841in which the vision system is identified as meeting standards, and is considered to be validated. The method of validating the visual acuity of a vision system is completed upon identifying the vision system as validated.

FIG.9is a process flow diagram which illustrates a method of assessing visual acuity, e.g., step829ofFIG.8A, in accordance with an embodiment. Method829of assessing visual acuity begins at a step809in which the data collected in step825ofFIG.8Ais obtained by an assessment system. Such an assessment system may be an application that runs on a computing system or at least one server.

The collected data is compared to a visual acuity benchmark for the vison testing medium in a step913. The visual acuity benchmark may include information that indicates what collected data is expected to indicate if the collected data meets at least minimum visual acuity standards. By way of example, if a vision testing medium is a Landolt eye chart, a visual acuity benchmark may include an indication of which Landolt C's of the Landolt eye chart an individual may be able to accurately read if at least a minimum visual acuity standard is to be considered to be met. Further, a percentage of correctly identified optotypes, e.g., Landolt C's, with respect to a particular number of optotypes may be specified as a visual acuity benchmark, e.g., identifying approximately seventy percent or at least approximately seven out of ten optotypes correctly may be specified as a benchmark for a substantially minimum acceptable visual acuity.

Based on the comparison of data, the visual acuity of the individual is determined in a step917. Such a determination may generally specify whether the individual meets at least minimum visual acuity standards. In one embodiment, if the individual is determined to meet at least minimum visual acuity standards, a vision system may be considered to also meet at least minimum visual acuity standards. After the visual acuity of the human is determined, the process of assessing visual acuity is completed.

As mentioned above, the visual acuity of a human may be assessed, and the assessment of the visual acuity of the human may be used to effectively validate a vision system of an autonomous vehicle.FIG.10is a diagrammatic representation of a system which may be used to assess visual acuity of a human using a teleoperations system in accordance with an embodiment. Human or individual556may view an image on a display system, e.g., a display screen included in display system446bof teleoperation system446. An assessment system866is arranged to collect or to otherwise obtain data from individual556that relates to what individual556sees displayed on display system446b. For example, if display system446bdisplays an eye test chart such as a Landolt eye chart, then assessment system866may collect information relating to how accurately individual556sees a gap in each Landolt C on the Landolt eye chart.

Assessment system1066generally includes a communications system1066a, a processing system1066b, and an optional data store1066c. Communications system1066a, which may include an input/output (I/O) interface, is generally arranged to obtain data from individual556. Communications system1066amay be arranged to communicate with individual556over a network, e.g., with a device such as a smartphone (not shown) used by individual556to transmit information to assessment system1066, and may be arranged to generally communicate with other nodes of a network, e.g., with a database on which data such as visual acuity data is stored.

The I/O interface of communications system1066amay vary widely. The I/O interface may be configured as a graphical user interface into which individual556, or another person, may enter data relating to what her or she is able to clearly and sharply discern from an image displayed on display system446. The I/O interface may instead, or additionally, be configured to receive audio and/or visual information that is collected from individual556as individual556speaks and/or gestures, e.g., verbalizes and/or gestures to indicate whether a Landolt C has a gap facing up, down, right, or left. It should be appreciated that data collected from individual556may be obtained indirectly from individual556, as for example from a database into which the data has been entered.

Processing system1066bis arranged to process data collected from individual556using communications system866ato assess the visual acuity of individual556. In one embodiment, the collected data may be compared with benchmark data stored in optional data store1066c. In another embodiment, the collected data may be compared with benchmark data obtained from a network using communications system1066a. Processing the data includes, but is not limited to including, assessing the data to determine whether the data indicates that individual556, when viewing an image displayed on display system446b, meets at least a minimum standard for visual acuity. When the data indicates that at least a minimum standard for visual acuity is achieved, a vision system such as vision system326ofFIG.3that provides the viewed image may be substantially classified as meeting at least the minimum standard for visual acuity.

Factors used to determine whether individual556and, hence, a vision system such as vision system326ofFIG.3, may generally vary widely. In one embodiment, individual556may be identified as meeting at least a minimum standard for visual acuity if he or she is able to correctly identify at least a predetermined number of optotypes such as a Landolt C out of a particular number of optotypes. example, if individual556is able to identify at least seven Landolt C characters out of ten Landolt C characters, individual556and, hence, a vision system such as vision system326of FIG.3may be substantially identified as meeting at least a minimum standard for visual acuity. It should be appreciated that different sensors within a vision system may effectively be evaluated to determine the visual acuity associated with each sensor.

A visual acuity assessment which utilizes an eye chart such as a Landolt C eye chart is not limited to being associated with a teleoperation system. That is, a visual acuity assessment may effectively be leveraged for use with systems other than teleoperation systems. By way of example, a visual acuity assessment may be performed to calibrate a simulation system configured to facilitate the training to teleoperation operators. In other words, the visual acuity of a simulation system may be assessed such that the overall performance of a system which allows for the simulation of driving environments may be evaluated. Assessing the visual acuity may be associated with substantially verify visual realism. It should be understood that a simulation system may be used to validate safety, e.g., end-to-end safety, of a teleoperations system.

FIG.11is a diagrammatic representation of a system which may be used to assess visual acuity of a teleoperations system and a simulation system in accordance with an embodiment. A human or individual1156may view an image on a display system, e.g., a display screen associated with a vison system1170of a simulation system1168. The image may be part of a recorded video. Individual1156may also view an image, e.g., a substantially live video image, on display system446bof teleoperation system446. In one embodiment, vison system1170may cause an image to be displayed on a display screen of display system,446b. It should be appreciated that an image rendered by display system446band an image rendered by vision system1170may be viewed by individual1156at different times.

Simulation system1168may include hardware and/or software which allows actual driving on roads to be substantially simulated. For example, video obtained by sensors or an autonomous vehicle may be recorded and rendered within simulation system1168such that individual1156may view the video when the video is rendered by vision system1170, although it should be appreciated that a video may instead be substantially fully synthetic, e.g., effectively generated without real-world sensor data or video. The video rendered by vision system1170may be displayed to individual1156, as for example on display screen446bor a separate display screen associated with simulation system116. In one embodiment, vision system1170may provide a rendering of an eye test chart such as a Landolt eye chart that individual1156may view.

An assessment system1066is arranged to collect or to otherwise obtain data from individual1156that relates to what individual1156sees displayed on display system446b. For example, if display system446bdisplays an eye test chart such as a Landolt eye chart, then assessment system1066may collect information relating to how accurately individual1156sees a gap in each Landolt C on the Landolt eye chart. Further, if vision system1170provides a rendering of a Landolt eye chart that is displayed to individual1156, assessment system1066may collect information relating to how accurately individual1156sees a gap in each Landolt C on the Landolt eye chart.

Assessment system1066includes communications system1066a, processing system1066b, and optional data store1066c. Assessment system1066may be remote with respect to teleoperation system446and simulation system1168, although it should be appreciated that assessment system1066may be local with respect to either or both teleoperation system446and simulation system1168.

Communications system1066a, which may include an input/output I/O interface, is generally arranged to obtain data from individual1156. Communications system1066amay be arranged to communicate with individual1156over a network, e.g., with a device such as a smartphone (not shown) used by individual556to transmit information to assessment system1066, and may be arranged to generally communicate with other nodes of a network, e.g., with a database on which data such as visual acuity data is stored. Alternatively, communication system1066amay communicate with individual1156through teleoperation system446and/or simulation system1168.

Processing system1066bis arranged to process data collected from individual1156using communications system1066ato assess the visual acuity of individual1156with respect to what individual1156sees with respect to display system446band/or vision system1170. In one embodiment, the collected data may be compared with benchmark data stored in optional data store1066c. In another embodiment, the collected data may be compared with benchmark data obtained from a network using communications system1066a. It should be appreciated that data obtained from user with respect to display system446bmay effectively be used as benchmark data for comparison purposes when data is obtained with respect to vision system1170. Processing the data includes, but is not limited to including, assessing the data to determine whether the data indicates that individual1156, when viewing an image displayed on display system446b, meets at least a minimum standard for visual acuity. When the data indicates that at least a minimum standard for visual acuity is achieved, a vision system such as vision system326ofFIG.3that provides the viewed image may be substantially classified as meeting at least the minimum standard for visual acuity.

Once vision system326is classified as meeting at least a minimum standard for visual acuity, data obtained from individual1156when individual1156views an image rendered by vision system1170may be processed to ascertain whether vision system1170meets visual acuity standards, e.g., at least a minimum standard for visual acuity of a simulation system. Such an assessment may include, but is not limited to including, determining whether individual1156views an eye chart rendered by display system446with approximately the same visual acuity with which individual1156views an eye chart rendered by vision system1170. For example, simulation system1168may be identified as substantially calibrated and as meeting visual acuity standards when the visual acuity associated with simulation system1168is approximately same as the visual acuity associated with teleoperations system446.

With reference toFIG.12, a method of calibrating a simulation system for use in enabling an individual to simulate driving a vehicle in real conditions that includes an assessment of visual acuity in accordance with an embodiment. A method1205of calibrating a simulation system begins at a step1209in which it is verified that an individual has vision that is within a range of acceptable visions. The simulation system may generally be a system that enables driving to be simulated, e.g., a system that allows a teleoperation system to be used to enable an operator to practice driving using a simulated scene or a video recording of an actual scene.

In a step1213, visual data, including an eye chart such as a Landolt eye chart, is rendered on a video system. The video system may be part or, or may be in communication with, a simulation system. Rendering the visual data on a video system may include, but is not limited to including, displaying the visual data on a display screen. The visual data may be viewed or otherwise seen by an individual, and information relating to the visual data may be collected from the individual in a step1217. The visual data may be collected using any suitable method including, but not limited to including, the individual inputting the data into an assessment system, the individual verbalizing what he or she sees and providing the information to an assessment system, and/or the individual providing an indication or what he or she sees directly or indirectly to an assessment system.

After information is collected from the individual, the collected information is analyzed in a step1221. Analyzing the collected information may include comparing the collected information with data to determine whether the collected information indicates that the simulation system is properly calibrated. One method of analyzing collected information will be discussed below with reference toFIG.13.

From step1221, process flow moves to a step1225in which it is determined whether the collected information indicates that the simulation system is calibrated. That is, it is determined whether the visual acuity indicated by the information is at an acceptable level. If the determination is that the simulation system is calibrated, then in a step1233, an indication that the simulation system is calibrated is provided, and the method of calibrating a simulation system is completed.

Alternatively, if it is determined in step1225that the information does not indicate that the simulation system is calibrated, the implication is that parameters associated with the simulation system may need to be adjusted or otherwise tuned. Accordingly, in a step1229, parameters associated with the simulation system are adjusted to effectively calibrate the simulation system. Parameters that may be adjusted may include, but are not limited to including, anti-aliasing parameters, color correction parameters, visual resolution parameters, lighting parameters, and/or the like. In one embodiment, adjusting parameters associated with a simulation system may involve identifying parameters associated with a teleoperations system and effectively matching the parameters of the simulation system with the parameters of the teleoperations system. Once parameters are adjusted, process flow returns to step1213in which visual data is displayed on the video system.

With reference toFIG.13, one method of analyzing collected information associated with a simulation system, e.g., step1221ofFIG.12, will be described in accordance with an embodiment. Method1221of analyzing collected information begins at a step1305in which data associated with the validation of a teleoperation system is obtained, e.g., from a data store such as data store1066cofFIGS.10and11. The data may be associated with the same individual who provided the collected information. In other words, the collected information may be collected from the same individual who was effectively used to validate the teleoperation system.

The collected information is compared with the data from the validation of the teleoperation system in step1309. Then, in a step1313, it is identified whether the collected information is consistent with the data from the validation of the teleoperation system. That is, it is determined whether the visual acuity or quality associated with the simulation system is similar to, or otherwise approximately the same as, the visual acuity or quality associated with the teleoperation system. In one embodiment, determining whether the collected information is consistent with the data from the validation of the teleoperation system includes determining whether a visual acuity associated with the teleoperations system. In another embodiment, determining whether the collected information is consistent with the data from the validation of the teleoperation system may involve determining whether the collected information indicates an acceptable level of visual acuity. For example, if the collected information indicates the ability to view an eye chart at approximately a 20/200 vision level, the implication may be that the collected information is consistent with the data from the validation of the teleoperation system. Upon identifying whether the collected information is consistent with the data from the validation of the teleoperation system, the method of analyzing collected information is completed.

While the use of a Landolt C eye chart has been described as being used to assess visual acuity, it should be understood that the use of eye charts is not limited to a Landolt C eye chart. For example, a Snellen eye chart may be used in some instance.FIG.14is a diagrammatic representation of a system which facilitates the validation of a vision system of a vehicle using a Snellen eye chart and a human in accordance with an embodiment. A vision test medium1468, which may be a Snellen eye test chart may be positioned within range of vision system326′ of autonomous vehicle101. Vision system326′ may provide an image of Snellen eye chart1368to display system446bof teleoperations system446, and a human1456may view the image to determine how clearly and sharply he or she is able to see the image, or at least the portion of the image which depicts Snellen eye chart1468. If human1456is able to clearly and sharply discern Snellen eye chart1468, e.g., read lines of optotype such as letters on Snellen eye chart1468, when viewed on display system446b, then vision system326′ may effectively be validated.

Referring next toFIG.15, an overall system arranged to evaluate the visual acuity or quality associated with a teleoperation system and a simulation system will be described in accordance with an embodiment. An overall system1570includes a teleoperation system1546, a simulation system1568, and an assessment system1566. Teleoperation system1546includes a control arrangement1572, which is generally configured to enable a vehicle or a robotic apparatus to be controlled, and a display screen1574. Control arrangement1572may include, but is not limited to including, a steering wheel or apparatus, speed control components, a gear shift, and a communication system. Display screen1574may be part of a display system, and may be a monitor arranged to display the environment around a vehicle or robotic apparatus that may be remotely controlled using teleoperations system1546.

Simulation system1568includes hardware and/or software configured to provide a simulated environment that may be used, as for example by teleoperation system1546, to essentially practice driving. The simulated environment may include, but is not limited to including, images such as video images of real environments through which a vehicle or robotic apparatus may be driven. In one embodiment, a visual rendering of an environment may be provided by simulation system1568to display screen1574such that control arrangement1572may be used to effectively simulate driving in the environment. That is, control arrangement1572may be configured to enable teleoperation system1546to be used to simulate driving in an environment provided by simulation system1568.

Assessment system1566is configured to obtain data associated with teleoperation system1546and data associated with simulation system1568to assess the visual acuity of teleoperation system1546and/or simulation system1568, Using the data, assessment system1566may determine whether visual acuity standards or expectations are met. In one embodiment, assessment system1566may determine whether the visual acuity associated teleoperation system1546and the visual acuity associated with simulation system1568are consistent with each other, e.g., are approximately the same.

Although only a few embodiments have been described in this disclosure, it should be understood that the disclosure may be embodied in many other specific forms without departing from the spirit or the scope of the present disclosure. By way of example, a vision system may be independently tested using the methods mentioned above. That is, the visual acuity of a vision system may be tested and, hence, validated even when the vision system is not installed on an autonomous vehicle. By testing visual acuity of a vision system before the vision system is installed on an autonomous vehicle, adjustments may be made to the vision system that may be more readily made. In one embodiment, the visual acuity may be tested using a display system that is separate from a teleoperation system.

A Landolt C circle or ring has generally been described as including a gap that is positioned along a perimeter of the ring. While the gap has been described as being positioned either at approximately zero degrees, approximately 90 degrees, approximately 180 degrees, or approximately 270 degrees along the ring, it should be appreciated that the gap may generally be positioned substantially anywhere along the ring. For instance, the gap may be positioned at approximately 45 degrees, approximately 135 degrees, approximately 225 degrees, or approximately 315 degrees along the ring.

While visual acuity standards have generally been specified in terms of what an average person may see on a vision eye chart when he or she is positioned at a predetermined distance away from the vision eye chart, it should be appreciated that other visual acuity standards may be implemented. In addition, visual acuity tests and/or vision test mediums are not limited to being used to facilitate a determination what a human, e.g., a teleoperator, is able to clearly see from a predetermined distance.

Visual acuity standards may generally involve determining whether an individual with substantially “perfect” vision is able to see a vision eye chart from a particular distance. Substantially perfect vision may be defined, for example, as approximately 20/20 vision in which an individual may clearly see and accurately read an eye chart from approximately twenty feet away or approximately 6/6 vision in which an individual may clearly see and accurately read an eye chart from approximately six meters away. It should be appreciated that the definition of perfect vision may vary widely, and that in some situations, an individual with less than approximately 20/20 vision, e.g., an individual with 20/40 vision, may be identified as having substantially perfect or otherwise acceptable vision.

In general, when a vision system of an autonomous vehicle is determined not to pass a visual acuity test, adjustments may be made to the vision system, and further visual acuity assessments may be made to determine if the vision system may eventually successfully pass a visual acuity test. Adjustments may include, but are not limited to including, altering native camera resolutions for cameras in the vision system, altering circuitry onboard the autonomous vehicle to reduce any degradation in resolution, installing new cameras in the vision system, substantially optimizing the parameters associated with video processing algorithms to provide improved image output, increasing bandwidth allocations, and/or changing the field of view associated with the vision system to focus more on a region of interest. For example, if a vision system of an autonomous vehicle is determined not to pass a visual acuity test, if image processing onboard the vehicle is identified as contributing to a degradation of resolution, adjustments may be made to algorithms, bandwidth allocations, cameras, etc. to improve the resolution.

As previously mentioned, different sensors within a vision system may effectively be evaluated to determine the visual acuity associated with each sensor. In one embodiment, the visual acuities associated with multiple sensors in a vision system may be evaluated such that the vision system, overall, may be considered to provide adequate visual acuity for teleoperations substantially only when each of the multiple sensors individual provides at least a minimum acceptable level of visual acuity. It should be appreciated, however, that as some sensors of a vision system may be considered to be more critical than others, when the visual acuities associated with the more critical sensors are substantially all considered to meet at least a minimum acceptable level of visual acuity, then the vision system may be considered to provide at least a minimum acceptable level of visual acuity.

An individual has been described as being a part of an overall system which allows for visual acuity of a display system of a teleoperation system to be assessed and/or for visual acuity of a vision system of a simulation system to be assessed. In one embodiment, in lieu of an individual, mechanisms may instead be used to assess visual acuity of a display system and/or a vision system. Such mechanisms may scan an eye chart that is rendered on display screen, and attempt to identify optotypes on the eye chart. The accuracy with which the optotypes are obtained and identified by a mechanism, e.g., a scanner or scanning device, may be used to determine visual acuity.

The calibration of a simulation system, as described above with respect toFIG.12, may be a continual process. For example, even in the event that an acceptable level of calibration with a current set of views from a camera is achieved, a change of camera views may result in a need to substantially re-calibrate the simulation system. Such a change in camera views may include, but is not limited to including, changing from a substantially standard camera view to a panoramic camera view.

An autonomous vehicle has generally been described as a land vehicle, or a vehicle that is arranged to be propelled or conveyed on land. It should be appreciated that in some embodiments, an autonomous vehicle may be configured for water travel, hover travel, and or/air travel without departing from the spirit or the scope of the present disclosure. In general, an autonomous vehicle may be any suitable transport apparatus that may operate in an unmanned, driverless, self-driving, self-directed, and/or computer-controlled manner.

The embodiments may be implemented as hardware, firmware, and/or software logic embodied in a tangible, i.e., non-transitory, medium that, when executed, is operable to perform the various methods and processes described above. That is, the logic may be embodied as physical arrangements, modules, or components. For example, the systems of an autonomous vehicle, as described above with respect toFIG.3, may include hardware, firmware, and/or software embodied on a tangible medium. In addition, an assessment system and a simulation system may include hardware, firmware, and/or software embodied on a tangible medium. A tangible medium may be substantially any computer-readable medium that is capable of storing logic or computer program code which may be executed, e.g., by a processor or an overall computing system, to perform methods and functions associated with the embodiments. Such computer-readable mediums may include, but are not limited to including, physical storage and/or memory devices. Executable logic may include, but is not limited to including, code devices, computer program code, and/or executable computer commands or instructions.

It should be appreciated that a computer-readable medium, or a machine-readable medium, may include transitory embodiments and/or non-transitory embodiments, e.g., signals or signals embodied in carrier waves. That is, a computer-readable medium may be associated with non-transitory tangible media and transitory propagating signals.

The steps associated with the methods of the present disclosure may vary widely. Steps may be added, removed, altered, combined, and reordered without departing from the spirit of the scope of the present disclosure. Therefore, the present examples are to be considered as illustrative and not restrictive, and the examples are not to be limited to the details given herein, but may be modified within the scope of the appended claims.