Package: ask-bedrock
Ask Amazon BedrockConverse with your favoriteAmazon Bedrocklarge language model from the command line.This tool is a wrapper around the low-level Amazon Bedrock APIs andLangchain. Its main added value is that it locally persists AWS account and model configuration to enable quick and easy interaction.Installation⚠️ Requires Python >= 3.9⚠️ Requires a working AWS CLI setup configured with a profile that allows Amazon Bedrock access. SeeCLI documentationfor details.pipinstallask-bedrockYou can also build/run this project locally, seeBuilding and Running Locally.UsageActivating modelsBefore you can use this command line tool, you need torequest model access through the AWS Consolein a region where Bedrock is available: Switch to the region where you want to run Bedrock, go to ”Model access“, click “Edit”, activate the models you wish to use, and then click “Save changes”.InvocationTo start a conversation, simply enter the following command:ask-bedrockconverseUpon the first run, you will be led through a configuration flow. To learn more about configuration options, see theConfigurationsection below.If you’re fully configured, the tool will show you a>>>prompt and you can start interacting with the configured model.Multi-line prompts can be wrapped into<<< >>>blocks.To end your interaction, hitCtrl + D. Note that the conversation will be lost.PricingNote that using Ask Amazon Bedrock incurs AWS fees. For more information, seeAmazon Bedrock pricing. Consider using a dedicated AWS account andAWS Budgetsto control costs.ConfigurationAsk Amazon Bedrockstores your user configuration in$HOME/.config/ask-bedrock/config.yaml. This file may contain several sets of configuration (contexts). For instance, you can use contexts to switch between different models. Use the--contextparameter to select the context you'd like to use. The default context isdefault.If no configuration is found for a selected context, a new one is created. If you want to change an existing config, useask-bedrockconfigure--contextmycontextYou can also create or edit the configuration file yourself in$HOME/.config/ask-bedrock/config.yaml:contexts:default:region:""# an AWS region where you have activated Bedrockaws_profile:""# a profile from your ~/.aws/config filemodel_id:""# a Bedrock model, e.g. "ai21.j2-ultra-v1"model_params:"{}"# a JSON object with parameters for the selected modelModel parametersThis JSON is passed to Langchain during client setup (asmodel_kwargs). The schema depends on the model that is used. Have a look at theexamples.If you want to configure multiple lines, model parameters can be wrapped in<<< >>>.Building and Running Locallypip install build
python -m build
python ask_bedrock/main.py converseFeedbackAs this tool is still early stage, we are very interested in hearing about your experience. Please take one minute to take a little survey:https://pulse.aws/survey/GTRWNHT1TroubleshootingQ:I’m getting the following error during invocation: “ValueError: Error raised by bedrock service: An error occurred (AccessDeniedException) when calling the InvokeModel operation: Your account is not authorized to invoke this API operation.”A:You may have selected a model that is currently not yet activated for public usage. It may have been listed it in the selection of available models, but unfortunately some models (such as Amazon Titan) aren’t yet available via API.Q:The model responses are cut off mid-sentence.A:Configure the model to allow for longer response. Use model parameters (see above) for this. Claude for example would take the following model parameters:{"max_tokens_to_sample": 3000}Q: I'm getting an error that is not listed here.A: Use the--debugoption to find out more about the error. If you cannot solve it, create an issue.SecuritySeeCONTRIBUTINGfor more information.LicenseThis project is licensed under the Apache-2.0 License.