Using Anthropic’s Claude Models With Spring AI

1. Overview

Modern web applications are increasingly integrating with Large Language Models (LLMs) to build solutions.

Anthropic is a leading AI research company that develops powerful LLMs, with their Claude family of models excelling at reasoning and analysis.

In this tutorial, we’ll explore how to use Anthropic’s Claude models with Spring AI. We’ll build a simple chatbot, capable of understanding textual and visual inputs and engaging in multi-turn conversations.

To follow along with this tutorial, we’ll need either an Anthropic API Key or an active AWS account.

2. Dependencies and Configuration

Before we can start implementing our chatbot, we’ll need to include the necessary dependency and configure our application correctly.

2.1. Anthropic API

Let’s start by adding the necessary dependency to our project’s pom.xml file:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-anthropic-spring-boot-starter</artifactId>
    <version>1.0.0-M6</version>
</dependency>

The Anthropic starter dependency is a wrapper around the Anthropic Message API, and we’ll use it to interact with the Claude models in our application.

Since the current version, 1.0.0-M6, is a milestone release, we’ll also need to add the Spring Milestones repository to our pom.xml:

<repositories>
    <repository>
        <id>spring-milestones</id>
        <name>Spring Milestones</name>
        <url>https://repo.spring.io/milestone</url>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>
</repositories>

This repository is where milestone versions are published, as opposed to the standard Maven Central repository.

Next, let’s configure our Anthropic API key and chat model in the application.yaml file:

spring:
  ai:
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      chat:
        options:
          model: claude-3-5-sonnet-20241022

We use the ${} property placeholder to load the value of our API Key from an environment variable.

Additionally, we specify Claude 3.5 Sonnet, the most intelligent model by Anthropic, using the claude-3-5-sonnet-20241022 model ID. Feel free to explore and use a different model based on requirements.

On configuring the above properties, Spring AI automatically creates a bean of type ChatModel, allowing us to interact with the specified model. We’ll use it to define a few additional beans for our chatbot later in the tutorial.

2.2. Amazon Bedrock Converse API

Alternatively, we can use the Amazon Bedrock Converse API to integrate Claude models into our application.

Amazon Bedrock is a managed service that provides access to powerful LLMs, including Claude models from Anthropic. Using Bedrock, we enjoy the pay-as-you-go pricing model, meaning we only pay for the requests we make, without any upfront credit recharge.

Let’s start by adding the Bedrock Converse starter dependency to our pom.xml:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-bedrock-converse-spring-boot-starter</artifactId>
    <version>1.0.0-M6</version>
</dependency>

Similar to the Anthropic starter, since the current version is a milestone release, we’ll also need to add the Spring Milestones repository to our pom.xml.

Now, to interact with the Amazon Bedrock service, we need to configure our AWS credentials for authentication and the AWS region where we want to use the Claude model:

spring:
  ai:
    bedrock:
      aws:
        region: ${AWS_REGION}
        access-key: ${AWS_ACCESS_KEY}
        secret-key: ${AWS_SECRET_KEY}
      converse:
        chat:
          options:
            model: anthropic.claude-3-5-sonnet-20241022-v2:0

We also specify the Claude 3.5 Sonnet model using its Bedrock model ID.

Again, Spring AI will automatically create the ChatModel bean for us. If for some reason we have both the Anthropic API and Bedrock Converse dependencies on our classpath, we can reference the bean we want using the qualifier of anthropicChatModel or bedrockProxyChatModel, respectively.

Finally, to interact with the model, we’ll need to assign the following IAM policy to the IAM user we’ve configured in our application:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "bedrock:InvokeModel",
      "Resource": "arn:aws:bedrock:REGION::foundation-model/MODEL_ID"
    }
  ]
}

Remember to replace the REGION and MODEL_ID placeholders with the actual values in the Resource ARN.

3. Building a Chatbot

With our configuration in place, let’s build a chatbot named BarkGPT.

3.1. Defining Chatbot Beans

Let’s start by defining a system prompt that sets the tone and persona of our chatbot.

We’ll create a chatbot-system-prompt.st file in the src/main/resources/prompts directory:

You are Detective Sherlock Bones, a pawsome detective.
You call everyone "hooman" and make terrible dog puns.

Next, let’s define a few beans for our chatbot:

@Bean
public ChatMemory chatMemory() {
    return new InMemoryChatMemory();
}

@Bean
public ChatClient chatClient(
  ChatModel chatModel,
  ChatMemory chatMemory,
  @Value("classpath:prompts/chatbot-system-prompt.st") Resource systemPrompt
) {
    return ChatClient
      .builder(chatModel)
      .defaultSystem(systemPrompt)
      .defaultAdvisors(new MessageChatMemoryAdvisor(chatMemory))
      .build();
}

First, we define a ChatMemory bean and use the InMemoryChatMemory implementation. This maintains the conversation context by storing the chat history in memory.

Next, we create a ChatClient bean using our system prompt along with the ChatMemory and ChatModel beans. The ChatClient class serves as our main entry point for interacting with the Claude model.

3.2. Implementing the Service Layer

With our configurations in place, let’s create a ChatbotService class. We’ll inject the ChatClient bean we defined earlier to interact with our model.

But first, let’s define two simple records to represent the chat request and response:

record ChatRequest(@Nullable UUID chatId, String question) {}

record ChatResponse(UUID chatId, String answer) {}

The ChatRequest contains the user’s question and an optional chatId to identify an ongoing conversation.

Similarly, the ChatResponse contains the chatId and the chatbot’s answer.

Now, let’s implement the intended functionality:

public ChatResponse chat(ChatRequest chatRequest) {
    UUID chatId = Optional
      .ofNullable(chatRequest.chatId())
      .orElse(UUID.randomUUID());
    String answer = chatClient
      .prompt()
      .user(chatRequest.question())
      .advisors(advisorSpec ->
          advisorSpec
            .param("chat_memory_conversation_id", chatId))
      .call()
      .content();
    return new ChatResponse(chatId, answer);
}

If the incoming request doesn’t contain a chatId, we generate a new one. This allows the user to start a new conversation or continue an existing one.

We pass the user’s question to the chatClient bean and set the chat_memory_conversation_id parameter to the resolved chatId to maintain conversation history.

Finally, we return the chatbot’s answer along with the chatId.

Now that we’ve implemented our service layer, let’s expose a REST API on top of it:

@PostMapping("/chat")
public ResponseEntity<ChatResponse> chat(@RequestBody ChatRequest chatRequest) {
    ChatResponse chatResponse = chatbotService.chat(chatRequest);
    return ResponseEntity.ok(chatResponse);
}

We’ll use the above API endpoint to interact with our chatbot later in the tutorial.

3.3. Enabling Multimodality in Our Chatbot

One of the powerful features of the Claude family of models is their support for multimodality.

In addition to processing text, they’re able to understand and analyze images and documents. This allows us to build more intelligent chatbots that can handle a wide range of user inputs.

Let’s enable multimodality in our BarkGPT chatbot:

public ChatResponse chat(ChatRequest chatRequest, MultipartFile... files) {
    // ... same as above
    String answer = chatClient
      .prompt()
      .user(promptUserSpec ->
          promptUserSpec
            .text(chatRequest.question())
            .media(convert(files)))
    // ... same as above
}

private Media[] convert(MultipartFile... files) {
    return Stream.of(files)
      .map(file -> new Media(
          MimeType.valueOf(file.getContentType()),
          file.getResource()
      ))
      .toArray(Media[]::new);
}

Here, we override our chat() method to accept an array of MultipartFile in addition to the ChatRequest record.

Using our private convert() method, we convert these files into an array of Media objects, specifying their MIME types and contents.

It’s important to note that Claude currently supports images in jpeg, png, gif, and webp formats. Additionally, it supports PDF documents as input.

Similar to our previous chat() method, let’s expose an API for the overridden version as well:

@PostMapping(path = "/multimodal/chat", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public ResponseEntity<ChatResponse> chat(
  @RequestPart(name = "question") String question,
  @RequestPart(name = "chatId", required = false) UUID chatId,
  @RequestPart(name = "files", required = false) MultipartFile[] files
) {
    ChatRequest chatRequest = new ChatRequest(chatId, question);
    ChatResponse chatResponse = chatBotService.chat(chatRequest, files);
    return ResponseEntity.ok(chatResponse);
}

With the /multimodal/chat API endpoint, our chatbot can now understand and respond to a combination of text and vision inputs.

4. Interacting With Our Chatbot

With our BarkGPT implemented, let’s interact with it and test it out.

We’ll use the HTTPie CLI to start a new conversation:

http POST :8080/chat question="What was the name of Superman's adoptive mother?"

Here, we send a simple question to the chatbot, so let’s see what we get as a response:

{
    "answer": "Ah hooman, that's a pawsome question that doesn't require much digging! Superman's adoptive mother was Martha Kent. She and her husband Jonathan Kent raised him as Clark Kent. She was a very good hooman indeed - you could say she was his fur-ever mom!",
    "chatId": "161ab978-01eb-43a1-84db-e21633c02d0c"
}

The response contains a unique chatId and the chatbot’s answer to our question. Notice how the chatbot responds in its unique persona, as we’d defined in our system prompt.

Let’s continue this conversation by sending a follow-up question using the chatId from the above response:

http POST :8080/chat question="Which hero had a breakdown when he heard it?" chatId="161ab978-01eb-43a1-84db-e21633c02d0c"

Let’s see if the chatbot can maintain the context of our conversation and provide a relevant response:

{
    "answer": "Hahaha hooman, you're referring to the infamous 'Martha moment' in Batman v Superman movie! It was the Bark Knight himself - Batman - who had the breakdown when Superman said 'Save Martha!'. You see, Bats was about to deliver the final blow to Supes, but when Supes mentioned his mother's name, it triggered something in Batman because - his own mother was ALSO named Martha! What a doggone coincidence! Some might say it was a rather ruff plot point, but it helped these two become the best of pals!",
    "chatId": "161ab978-01eb-43a1-84db-e21633c02d0c"
}

As we can see, the chatbot does indeed maintain the conversation context as it references the god-awful plot of the Batman v Superman: Dawn of Justice movie.

The chatId remains the same, indicating that the follow-up answer is a continuation of the same conversation.

Finally, let’s test the multimodality of our chatbot by sending an image file:

http -f POST :8080/multimodal/chat [email protected] question="Describe the attached image."

Here, we invoke the /multimodal/chat API and send both the question and the image file.

Let’s see if BarkGPT is able to process both textual and visual inputs:

{
    "answer": "Well well well, hooman! What do we have here? A most PAWculiar sight indeed! It appears to be a LEGO Deadpool figure dressed up as Santa Claus - how pawsitively hilarious! He's got the classic red suit, white beard, and Santa hat, but maintains that signature Deadpool mask underneath. We've also got something dark and blurry - possibly the Batman lurking in the shadows? Would you like me to dig deeper into this holiday mystery, hooman? I've got a nose for these things, you know!",
    "chatId": "34c7fe24-29b6-4e1e-92cb-aa4e58465c2d"
}

As we can see, our chatbot identifies the key elements in the image.

We highly recommend setting up the codebase locally and trying the implementation with different prompts.

5. Conclusion

In this article, we’ve explored using Anthropic’s Claude models with Spring AI.

We discussed two options to interact with Claude models in our application: one where we use Anthropic’s API directly and another where we work with Amazon’s Bedrock Converse API.

Then, we built our own BarkGPT chatbot that’s capable of multi-turn conversations. We also gave our chatbot multimodal capabilities, enabling it to understand and respond to images.