Spring AI: There and Back Again

Every great journey begins with a single step out the front door. For Spring developers, that step is a ChatClient and a prompt. Simple, safe, and surprisingly powerful. But the road goes ever on, and what starts as a quiet conversation with a language model quickly leads somewhere much bigger.

This is that adventure. We'll set out from the Shire with the basics of Spring AI and chart a course through stranger lands. We'll teach our application to use tools, the way any good traveler learns to use what's in their pack. We'll cross into the wider world through MCP, connecting our Spring Boot app to everything around it. And at the edge of the map, we'll meet the agents. Applications that can think, plan, and act on their own.

Along the way you'll pick up working code, a clear mental model of how the pieces of Spring AI fit together, and a sense of where each one earns its place in a real application. No toy examples. No waving at concepts from a distance. Just a Spring Boot app that grows up a little at every stop on the road.

You'll come back the way all good adventurers do. With stories to tell, a pack full of new tools, and a map for wherever you want to go next.

What You'll Learn

• The Shire: ChatClient, prompts, system messages, streaming, and structured output
• Bree: Giving the model tools it can call in your own Java code
• Rivendell: MCP, using other people's tools as a client and sharing your own as a server
• Mordor: Agents that plan, act, and change course on their own
• How tokens, context windows, and pricing shape what you build

Slides

1 / 61

PDFDownload PDF4.7 MB

Slide text

1. 1. SPRING AI There and Back Again Dan Vega · Spring Developer Advocate Grab the code while you settle in. The demo repo github.com/danvega/spring-ai-there-and-back-again
2. 2. SPRING AI There and Back Again Dan Vega Spring Developer Advocate
3. 3. Dan Vega Spring Developer Advocate Java Champion Author, Fundamentals of Software Engineering Cleveland, Ohio (my Shire) Find out more at danvega.dev
4. 4. SET TING THE SCOPE When we say AI, we mean two different things 01 02 AI that writes the code AI inside the code you ship Assistants and agents in the IDE Models called from your own services Writing, reviewing and debugging Features your users actually touch Reading unfamiliar codebases Your data, your rules, your uptime A different talk This talk
5. 5. You don't need to leave Java to build AI apps.
6. 6. Why are we talking about AI on the JVM?
7. 7. PY THON EARNED ITS PLACE IN AI The training era belonged to Python Python dominated AI research and model training. We are not here to debate that story. TensorFlow PyTorch Scikit-Learn But the game has changed. Hugging Face
8. 8. WHERE THE MONEY WENT Compute moved from training to running models 2/3 55% 74% of AI compute in 2026 is inference, up from a third in 2023 of AI cloud spend is inference, past training for the first time of builders say most of their workload is inference, up from 48% Gartner, 2026 Deloitte, TMT Predictions 2026 Menlo Ventures LLM market update Running models in production is application work, and application work is where Java lives.
9. 9. B U T W H AT A R E W E A C T U A L LY D O I N G ? You already own the tools for this job Training a model: a research problem Consuming a model: an integration problem Sourcing and cleaning a corpus Distributed training on a GPU cluster Calling the API RestClient, provider SDKs Weeks of runs, most of them discarded Evaluation and alignment before anyone can use it A handful of labs do this Most of us aren't training models. We're calling them. Credentials and config Spring Boot configuration, Vault Retries and timeouts Spring Retry, Resilience4j Observability and cost Micrometer, OpenTelemetry
10. 10. YOU ' V E D ONE T H I S B E FORE LLMs are just the next integration Databases → Message queues → Payment gateways → Cloud → LLMs
11. 11. WHY THE JVM Java is built for this 01 02 03 04 Enterprise-grade infrastructure Type safety & structured output Battle-tested at scale Your existing team Spring, Kafka and observability tooling already running in production at scale Parse AI responses into real objects with compile-time guarantees, not loose strings Millions of JVM applications serving billions of requests every single day No need to hire a Python shop or replatform. Upskill the developers you already trust
12. 12. Before the Framework
13. 13. Calling Claude with cURL /bin/bash ANTHROPIC_API_KEY="YOUR_API_KEY_HERE" PROMPT="Tell me an interesting fact about Java" / / curl https: api.anthropic.com/v1/messages \ -H "Content-Type: application/json" \ -H "x-api-key: $ANTHROPIC_API_KEY" \ -H "anthropic-version: 2023-06-01" \ -d '{"model": "claude-haiku-4-5", "max_tokens": 1024, "messages": [{"role": "user", "content": "'"${PROMPT}"'"}]}' ! # NO FRAMEWORK
14. 14. W H AT C O M E S B A C K Raw JSON, and your own parsing { "id": "msg_01XFDUDYJgAACzvnptvVoYEL", "type": "message", "role": "assistant", "model": "claude-haiku-4-5", "content": [ { "type": "text", "text": "Java was initially designed with interactive television in mind ], "stop_reason": "end_turn", "usage": { "input_tokens": 14, "output_tokens": 90 } . . . } " }
15. 15. NO FRAMEWORK Calling Claude with plain Java var body = """ { "model": "claude-haiku-4-5", "max_tokens": 1024, "messages": [{ "role": "user", "content": "Tell me an interesting fact about Java" }] } """; HttpRequest request = HttpRequest.newBuilder() .uri(URI.create("https: api.anthropic.com/v1/messages")) .header("Content-Type", "application/json") .header("x-api-key", apiKey) .header("anthropic-version", "2023-06-01") .POST(HttpRequest.BodyPublishers.ofString(body)) .build(); / / var client = HttpClient.newHttpClient(); var response = client.send(request, HttpResponse.BodyHandlers.ofString()); System.out.println(response.body());
16. 16. “ f Spring AI provides so much more than a acility for making REST API calls.
17. 17. W H AT A R E T H E C H A L L E N G E S ? Everything you would otherwise build yourself 01 02 03 Talking to models Running it in production Going past one call Model abstraction and portability Error handling and retry logic Structured output parsing Unified API across providers Observability and monitoring Prompt template management Multimodality: text, image, audio Token counts and cost control Memory and conversation state Streaming and async responses Evaluation and testing RAG, embeddings, vector stores Security and compliance Tool calling and MCP
18. 18. what is a token? You pay for tokens, not words 100 tokens is about 75 words. A token is roughly three quarters of a word. The model reads, writes and bills in tokens. The words you send and the words you get back both count. Remember the usage block on the raw JSON slide? input_tokens and output_tokens are the bill.
19. 19. T W O WAY S T O PAY ChatGPT pricing is not API pricing $/month $/token A subscription. Flat and predictable. ChatGPT Plus or Claude Pro is about $20 a month for one person in a chat app. The API. Metered, with no ceiling. Every request your app sends is billed by the token, in and out. What you pay when you chat What your Spring Boot app pays Your app uses the API. Tokens are your bill.
20. 20. pricing What the models cost, per million tokens Standard rates for short prompts, checked October 13, 2026. Long prompts, batch and cached input price differently, and intro prices end soon. Check the provider's page before you budget.
21. 21. context window One budget, spent on every call The model has no memory between calls. Everything below is sent again on every request, and it all has to fit inside the window. Run out of room and the call fails. Fill it carelessly and you pay for it on every turn.
22. 22. the catch Context rot More context is not better context. Chroma tested 18 models, including GPT - Chroma Research, July 2025
23. 23. The Road Ahead
24. 24. THE ROAD AHEAD Who decides what happens next? The Shire Bree Rivendell Mordor ChatClient Tools MCP Agents Your code decides The model decides
25. 26. “It's a dangerous business, going out your door.” BILBO BAGGINS Step one: start.spring.io
26. 27. How Spring Hands You a ChatClient pom.xml <dependency> <groupId>org.springframework.ai groupId> <artifactId>spring-ai-starter-model-anthropic dependency> artifactId> 1 Add one starter. application.yaml spring: ai: anthropic: api-key: ${SPRING_AI_ANTHROPIC_API_KEY} chat: model: claude-haiku-4-5 2 Set your key and model. ShireController.java public ShireController(ChatClient.Builder builder) { this.chatClient = builder.build(); } 3 Ask for it in your constructor. / < f / < No new. No setup code. That's auto-con iguration. / < CHAPTER ONE · THE SHIRE At startup, Spring sees both and builds an AnthropicChatModel and a ChatClient.Builder for you.
27. 28. MAKE CAMP · CHAPTER ONE Camp 1: The Shire WHAT WE'LL BUILD A new project from start.spring.io Your first prompt, then stream it A messy email into a Java record git checkout 01-shire
28. 29. W H AT W E J U S T B U I LT · T H E S H I R E Your First Prompt ShireController.java String answer = chatClient.prompt() .system("You are a hobbit who has never left the Shire.") .user("Plan a perfect day in the Shire, starting with second breakfast.") .call() .content();
29. 30. W H AT W E J U S T B U I LT · T H E S H I R E Streaming ShireController.java Wait for the whole answer .call().content() String Watch it arrive .stream().content() / / / / Same prompt. One word changes. Flux<String>
30. 31. W H AT W E J U S T B U I LT · T H E S H I R E Structured Output Journey.java · ShireController.java record Journey(String traveler, LocalDate departure, List<Stop> stops) {} record Stop(String place, LocalDate arrival, String lodging, String confirmationCode) {} Journey journey = chatClient.prompt() .system("Extract the journey details from this email.") .user(email) .call() .entity(Journey.class);
31. 32. Most AI features never need to leave the Shire.
32. 34. CHAPTER TWO · BREE How a Tool Call Works Your code runs on this side only. YOUR SPRING B O OT APP You Spring AI manages the loop 1 Sends your question and the tool list to Claude 3 Runs getWeather("Rivendell") in your app What should I pack for Rivendell? 4 Sends back: Cold and misty. Rain by evening. 6 Hands the final answer back to you Claude The model 2 getWeather("Rivendell") Not an answer. A request. 5 Pack a warm cloak and waterproof boots. Your Tool methods getCurrentDate() getWeather(place) The model never runs your code. It only asks. @ Spring AI runs this loop for you.
33. 35. MAKE CAMP · CHAPTER TWO Camp 2: Bree WHAT WE'LL BUILD A tool that knows today's date A tool that checks the weather Watch the model choose git checkout 02-bree
34. 36. Defining a Tool TravelTools.java @Tool(description = "Get today's date") String getCurrentDate() { return LocalDate.now().toString(); } @Tool(description = "Get the current weather for a place in Middle-earth") String getWeather(@ToolParam(description = "The place, for example Bree or Rivendell") String place) { Sunny in the Shire, misty in Rivendell, ash in Mordor } BreeController.java chatClient.prompt() .user(prompt) .tools(new TravelTools()) .call() .content(); / / W H AT W E J U S T B U I LT · B R E E
35. 37. CHAPTER TWO · BREE What do you trust the model to carry? Read Tools Write Tools Look things up. Change things. Weather, search, database queries Send email, update records, place orders S A F E TO C A RRY C A RRY W I T H C A RE
36. 39. One protocol to connect them all. modelcontextprotocol.io
37. 40. mcp by the numbers Everyone speaks MCP now 400M SDK downloads a month, up 4x this year Python, TypeScript, Java, Kotlin, C#, Go, Rust, Swift, Ruby, PHP + Anthropic, July 2026
38. 41. december 2025 Anthropic donates MCP to the Agentic AI Foundation A directed fund under the Linux Foundation. Neutral governance. It belongs to you now.
39. 42. MADE IN SPRING Java's MCP SDK came from the Spring team Nov 2024 Feb 2025 Today Spring AI starts an experimental MCP project for Java It becomes the official MCP Java SDK, built with Anthropic Spring AI 2.0 builds on it. A starter and a few lines of YAML. Spring blog, February 2025 When you use MCP from Java, you're using code the Spring team wrote.
40. 43. T W O WAY S T O C O N N E C T Local or remote, same protocol stdio Streamable HTTP The server runs on your machine. Your app starts it as a process and talks to it over standard input and output. The server runs anywhere on the network. Clients connect to it with a URL. In our demo, the app starts the filesystem server In our demo, the MCP Inspector connects to /mcp Same tools, same messages. Only the connection changes.
41. 44. CHAPTER THREE · RIVENDELL Client and Server YOUR A P P A S A C L I E N T Y O U R A P P A S A S E RV E R Any MCP client GitHub Claude Desktop Filesystem server uses its tools Your Spring Boot app Claude Code Databases Hundreds more t Your app uses tools other people buil .
42. 45. MAKE CAMP · CHAPTER THREE Camp 3: Rivendell WHAT WE'LL BUILD Use an existing MCP server Expose our tools as an MCP server Call them from Claude git checkout 03-rivendell
43. 46. W H AT W E J U S T B U I LT · R I V E N D E L L Your App as a Client application.yaml spring: ai: mcp: client: stdio: connections: filesystem: command: npx args: ["-y", "@modelcontextprotocol/server-filesystem" , "./journeys" ] RivendellController.java chatClient.prompt() .user(prompt) .tools(new TravelTools(), mcpTools) .call() .content(); it can only touch this one folder
44. 47. W H AT W E J U S T B U I LT · R I V E N D E L L Your App as a Server application.yaml spring: ai: mcp: server: name: journey protocol: STREAMABLE #required: without it you get the older SSE transport McpServerConfig.java @Bean ToolCallbackProvider travelTools() { return MethodToolCallbackProvider.builder() .toolObjects(new TravelTools()) .build(); } Building an MCP server from scratch? Use @McpTool.
45. 49. One does not simply walk into agents.
46. 50. Why didn't they just fly the eagles? t Planning is the hard par .
47. 51. CHAPTER FOUR · MORDOR How an Agent Works GOAL Plan the fastest safe route from Rivendell to Mount Doom. AC T · what it called C H E C K · what came back P L A N · what it decides next getRoutes(Rivendell) Caradhras 2 days, Moria 4 days Caradhras is faster. Check the weather first. getWeather(Caradhras) Blizzard. Pass blocked. Re-plan: go through Moria. getRoutes(Moria) Lothlorien, 2 days. "Something stirs in the deep." It's open. Keep going. requestEagles(Mount Doom) Declined. Walk. Keep walking. getRoutes(Dead Marshes) Black Gate closed. Cirith Ungol open. Take Cirith Ungol. write_file(route-to-mordor.md) Saved. Done. 22 days. ... a few more routes ... An agent is the tool loop from Bree, given a goal instead of a question.
48. 52. MAKE CAMP · CHAPTER FOUR Camp 4: Mordor WHAT WE'LL BUILD Give it a goal, not a question Watch it plan and act See where it goes wrong git checkout 04-mordor
49. 53. W H AT W E J U S T B U I LT · M O R D O R What We Just Built MordorController.java builder.defaultSystem(""" You are a route planner for Middle-earth. Reach the goal using your tools. Explore the routes one step at a time. Never invent a route. Check the weather before crossing any mountain pass. """) chatClient.prompt() .user(goal) .tools(new TravelTools(), new RouteTools(), mcpTools) .call() .content(); Same loop as Bree. A goal, more tools, and instructions. The model did the planning.
50. 54. BEYOND MORDOR Going Further: Embabel An agent framework for the JVM, from the creator of Spring Plans with an algorithm GOAP, from game AI , not the LLM Re-plans after every step Typed Java: @Agent, @Action, @AchievesGoal “Spring AI is the Servlet API. Embabel is Spring MVC.” the Embabel README ) ( github.com/embabel/embabel-agent
51. 55. CHAPTER FOUR · MORDOR Don't go further than you need to. Workflow: the road Agent: the wilderness You choose the path. The model chooses the path. Predictable. ◆ Cheaper. ◆ Easy to test. Flexible. ◆ Slower and costlier. ◆ Might walk into Moria. Most features are roads. Save agents for the wilderness.
52. 56. There and Back Again
53. 57. THERE AND BACK AGAIN Which stop do you need? The Shire Text goes in and text or data comes out. Summaries, extraction, classification. ChatClient & Prompts Bree Tools Rivendell MCP Mordor Agents The answer needs live data, or the app needs to act in your system. The tools already exist somewhere else, or other AI apps should use yours. You can't know the steps ahead of time, and the task is worth the cost and risk.
54. 59. COMING IN SPRING AI 2.1 The Road Goes Ever On THE SHIRE Session management. Conversation history is coming to Spring AI's core. BREE Message parts. Replies keep their order: reasoning, tool calls, text, and images. RIVENDELL The newest MCP spec 2026 07 28) is in progress. MORDOR Spring AI Agents. A new experimental project, planned for November 2026. - - ( - 2.1.0 M1 is out now. Milestone APIs may still change.
55. 60. THERE AND BACK AGAIN Your Next Steps 1 2 Spring AI reference docs.spring.io/spring-ai/reference Spring AI Community github.com/spring-ai-community Community projects that extend Spring AI 3 The demo repo github.com/danvega/spring-ai-there-and-back-again
56. 61. Thank You Dan Vega danvega.dev The demo repo github.com/danvega/spring-ai-there-and-back-again

Code