A system for using a multi-factor authentication process to access a network-accessible voice-based service comprising: a voice-capturing endpoint in communication with the voice-based service operating in a service provider environment; and one or more computing devices in the service provider environment in communication with the voice-based service, wherein the one or more computing devices include instructions that when executed: register a plurality of policies for allowing access to the voice-based service operating in the service provider environment, wherein the plurality of policies indicate levels of access based at least in part on an authenticated identity; receive an identifier from the voice-capturing endpoint, the ID associated with a device or user within proximity to the voice-capturing endpoint; generate the authenticated identity based at least in part on authenticating the user or device and voice data received from the voice-capturing endpoint; send the authenticated identity to the voice-capturing endpoint; receive a voice command with the authenticated identity, via the voice-capturing endpoint, to access the voice-based service operating in the service provider environment; and determine, based at least in part on the authenticated identity and the voice command, whether to execute the voice command according to a level of access indicated in at least one of the plurality of policies to allow access to the voice-based service. a voice-capturing endpoint in communication with the voice-based service operating in a service provider environment; and one or more computing devices in the service provider environment in communication with the voice-based service, wherein the one or more computing devices include instructions that when executed: register a plurality of policies for allowing access to the voice-based service operating in the service provider environment, wherein the plurality of policies indicate levels of access based at least in part on an authenticated identity; receive an identifier from the voice-capturing endpoint, the ID associated with a device or user within proximity to the voice-capturing endpoint; generate the authenticated identity based at least in part on authenticating the user or device and voice data received from the voice-capturing endpoint; send the authenticated identity to the voice-capturing endpoint; receive a voice command with the authenticated identity, via the voice-capturing endpoint, to access the voice-based service operating in the service provider environment; and determine, based at least in part on the authenticated identity and the voice command, whether to execute the voice command according to a level of access indicated in at least one of the plurality of policies to allow access to the voice-based service. register a plurality of policies for allowing access to the voice-based service operating in the service provider environment, wherein the plurality of policies indicate levels of access based at least in part on an authenticated identity; receive an identifier from the voice-capturing endpoint, the ID associated with a device or user within proximity to the voice-capturing endpoint; generate the authenticated identity based at least in part on authenticating the user or device and voice data received from the voice-capturing endpoint; send the authenticated identity to the voice-capturing endpoint; receive a voice command with the authenticated identity, via the voice-capturing endpoint, to access the voice-based service operating in the service provider environment; and determine, based at least in part on the authenticated identity and the voice command, whether to execute the voice command according to a level of access indicated in at least one of the plurality of policies to allow access to the voice-based service.