Use Anthropic Claude 3 models to build generative AI applications with Go
Tags: go, ai, programming
Published: 2024-05-14 06:55:27

Anthropic's Claude 3 is a family of AI models with different capabilities and cost for a variety of tasks:

- **Claude 3 Haiku** is a compact and fast model that provides near-instant responsiveness
- **Claude 3 Sonnet** provides a balance between  skills and speed 
- **Claude 3 Opus** is for highly complex tasks when you that need high intelligence and performance

Claude 3 models are multi-modal. This means that they can accept both text and images as inputs (although they can only output text). Let's learn how to use the Claude 3 models on Amazon Bedrock with Go.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/12xrlwg5hf1wrxx7ym9c.jpeg)

## Basic examples

Refer to **Before You Begin** section in [this blog post](https://community.aws/concepts/amazon-bedrock-golang-getting-started#before-you-begin) to complete the prerequisites for running the examples. This includes installing Go, configuring Amazon Bedrock access and providing necessary IAM permissions.

Amazon Bedrock abstracts multiple models via a uniform set of APIs that exchange JSON payloads. Same applies for Claude 3.

Let's start off with a simple example using [AWS SDK for Go (v2)](https://aws.github.io/aws-sdk-go-v2/docs/).

You can run it as such:

```shell
git clone https://github.com/abhirockzz/claude3-bedrock-go
cd claude3-bedrock-go

go run basic/main.go
```

The response may (or may not) be slightly different in your case:

```shell
request payload:
 {"anthropic_version":"bedrock-2023-05-31","max_tokens":1024,"messages":[{"role":"user","content":[{"type":"text","text":"Hello, what's your name?"}]}]}
response payload:
 {"id":"msg_015e3SJ99WF6p1yhGTkc4HbK","type":"message","role":"assistant","content":[{"type":"text","text":"My name is Claude. It's nice to meet you!"}],"model":"claude-3-sonnet-28k-20240229","stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":14,"output_tokens":15}}
response string:
 My name is Claude. It's nice to meet you!
```

You can refer to the [complete code here](https://github.com/abhirockzz/claude3-bedrock-go/blob/master/basic/main.go). I will break it down to make this simpler for you:

We start by creating the JSON payload - it's modeled as a `struct` (`Claude3Request`):

```go
	msg := "Hello, what's your name?"

	payload := Claude3Request{
		AnthropicVersion: "bedrock-2023-05-31",
		MaxTokens:        1024,
		Messages: []Message{
			{
				Role: "user",
				Content: []Content{
					{
						Type: "text",
						Text: msg,
					},
				},
			},
		},
	}

  payloadBytes, err := json.Marshal(payload)
```

[InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) is used to call the model. The JSON response is converted to a struct (`Claude3Response`) and the text response is extracted from it.

```go
//....
  output, err := brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
      Body:        payloadBytes,
      ModelId:     aws.String(modelID),
      ContentType: aws.String("application/json"),
  })

  var resp Claude3Response
  err = json.Unmarshal(output.Body, &resp)
  
  log.Println("response string:\n", resp.ResponseContent[0].Text)
```

## Chat with streaming

Now moving on to a common example which involves a conversational exchange. We will also add a streaming element to for better experience - the client application does not need to wait for the complete response to be generated for it start showing up in the conversation.

You can run it as such:

```shell
go run chat-streaming/main.go
```

> You can refer to the [complete code here](https://github.com/abhirockzz/claude3-bedrock-go/blob/master/chat-streaming/main.go).

A streaming based implementation is a bit more involved. First off, we use `InvokeModelWithResponseStream` (instead of `Invoke`).

```go
	output, err := brc.InvokeModelWithResponseStream(context.Background(), &bedrockruntime.InvokeModelWithResponseStreamInput{
		Body:        payloadBytes,
		ModelId:     aws.String(modelID),
		ContentType: aws.String("application/json"),
	})

	resp, err := processStreamingOutput(output, func(ctx context.Context, part []byte) error {
		fmt.Print(string(part))
		return nil
	})
```

To process it's output, we use:

```go
	resp, err := processStreamingOutput(output, func(ctx context.Context, part []byte) error {
		fmt.Print(string(part))
		return nil
	})
//...
```

Here are a few bits from the `processStreamingOutput` function - you can [check the code here](https://github.com/abhirockzz/claude3-bedrock-go/blob/master/chat-streaming/main.go#L206). The important thing to understand is how the partial responses are collected together to produce the final output.

```go
func processStreamingOutput(output *bedrockruntime.InvokeModelWithResponseStreamOutput, handler StreamingOutputHandler) (Claude3Response, error) {

  //...
	for event := range output.GetStream().Events() {
		switch v := event.(type) {
		case *types.ResponseStreamMemberChunk:

			var pr PartialResponse
			err := json.NewDecoder(bytes.NewReader(v.Value.Bytes)).Decode(&pr)
			if err != nil {
				return resp, err
			}

			if pr.Type == partialResponseTypeContentBlockDelta {
				handler(context.Background(), []byte(pr.Delta.Text))
				combinedResult += pr.Delta.Text
			} else if pr.Type == partialResponseTypeMessageStart {
				resp.ID = pr.Message.ID
				resp.Usage.InputTokens = pr.Message.Usage.InputTokens
			} else if pr.Type == partialResponseTypeMessageDelta {
				resp.StopReason = pr.Delta.StopReason
				resp.Usage.OutputTokens = pr.Message.Usage.OutputTokens
			}
  //...
	}

  resp.ResponseContent[0].Text = combinedResult

	return resp, nil
}
```

## Use Claude Haiku with images

All the Claude3 models can accept images as inputs. Haiku is deemed to be good at OCR (optical), understanding images, etc. Let's make use of Claude Haiku the upcoming examples - no major changes required, except for the model ID.

The example uses this image (actual StackOverflow question) along with this prompt `Transcribe the code in the question. Only output the code.`


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ysz1ciqwunho1380zte0.jpg)

You can run it as such:

```shell
go run images/main.go
```

This is the response that Claude Haiku came up with. Impressive!

```go
 func test() {
    cluster := gocql.NewCluster("hostIP")
    cluster.ProtoVersion = 4
    cluster.Authenticator = gocql.PasswordAuthenticator{
        Username: "<username>",
        Password: "<password>",
    }
    cluster.Keyspace = "myspace"
    cluster.Consistency = gocql.One
    session, err := cluster.CreateSession()
    if err != nil {
        panic(err)
    }
    defer session.Close()
}
```

I tried another one: "Can you suggest a solution to the question?"

And here is the response:

```shell
 Based on the error messages shown in the image, it seems that the issue is related to connecting to the Cassandra database using the gocql driver (Go: 1.8.3). The errors indicate that the application is unable to dial the control connection, create a session, or connect to the initial hosts within the timeout period.

A few things you could check to troubleshoot this issue:

1. Verify the connection details (host, port, username, password) are correct and match the Cassandra cluster configuration.

2. Ensure the Cassandra cluster is up and running and accessible from the application host.

3. Check the firewall settings to make sure the application host is able to connect to the Cassandra cluster on the required ports.

4. Inspect the Cassandra server logs for any errors or clues about the connection issue.

5. Try increasing the timeout values in the gocql configuration, as the current timeouts may be too short for the Cassandra cluster to respond.

6. Ensure the gocql version (1.8.3) is compatible with the Cassandra version (3.10.0) you are using.

7. Consider adding some error handling and retry logic in your code to gracefully handle connection failures and attempt to reconnect.

Without more context about your specific setup and environment, it's difficult to pinpoint the exact issue. However, the steps above should help you investigate the problem and find a solution.
```

> You can refer to the [complete code here](https://github.com/abhirockzz/claude3-bedrock-go/blob/master/images/main.go).

Here is what the JSON payload for the request:

```go
	payload := Claude3Request{
		AnthropicVersion: "bedrock-2023-05-31",
		MaxTokens:        1024,
		Messages: []Message{
			{
				Role: "user",
				Content: []Content{
					{
						Type: "image",
						Source: &Source{
							Type:      "base64",
							MediaType: "image/jpeg",
							Data:      imageContents,
						},
					},
					{
						Type:   "text",
						Text:   msg,
						Source: nil,
					},
				},
			},
		},
	}
```

The `imageContents` in `Data` attribute is the `base64` encoded version of the image which is calculated like this:

```go
func readImageAsBase64(filePath string) (string, error) {

	imageFile, err := os.ReadFile(filePath)

	if err != nil {
		return "", err
	}

	encodedString := base64.StdEncoding.EncodeToString(imageFile)
	return encodedString, nil
}
```

You can try a different image ([for example this one](https://github.com/abhirockzz/claude3-bedrock-go/blob/master/menu.jpg)), and check if it's able to calculate the cost of all menu items.

## Combine text and image conversations

Image and text data are not exclusive. The message structure (JSON payload) is flexible enough to support both.

> You can refer to the [complete code here](https://github.com/abhirockzz/claude3-bedrock-go/blob/master/multi-modal-chat-streaming/main.go).

Here is an example in which you can mix and match text and image based messages. You can run it as such:

```shell
go run multi-modal-chat-streaming/main.go
```

Here is the response (I used the same StackOverflow image):


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b0uoop10j7y7goh21hsx.png)

## Wrap up

You don't have to depend on Python to build generative AI applications. Thanks to AWS SDK support, you can use the programming language of your choice (including Go!) to integrate with Amazon Bedrock.