LLM integration

Calling a language model from application code with the tenant, destination and credential handling already solved instead using an SDK call with a hand-rolled key.

Overview

SAP SHIPSby design

Node.js →

@cap-js/ai Out-of-the-box value-help recommendations powered by SAP RPT-1, SAP AI Core as a standard CAP service, and an experimental local embedding model behind SQLite's vector functions.

Java →

cap-java/cds-ai Bridges CAP Java to SAP AI Core and adds AI-powered field recommendations to Fiori draft UIs, as two alpha Maven modules with mock fallbacks for local development.

Not covered: It stops at models reached through SAP AI Core: every package above is a client of that service, and aicore is the kind cds.requires.llm defaults to in production and hybrid. The other kinds capire documents are for local runs, auto and mock on Node.js and ollama and mocked on Java, and the page that configures the Java one marks chat model configuration alpha.

SourceCAP plugins — AIchecked 2026-10-15

THE COMMUNITY ADDS

@saptarishi/cds-plugin-llm puts many model providers behind one CAP service kind, so which backend a handler calls is a profile rather than a code change, and cap-llm-plugin anonymises the sensitive fields in SAP HANA Cloud before the prompt leaves. @geertjanklaps/cds-toon-ai-protocol-adapter leaves the call alone and changes the wire format instead, so the same rows cost fewer tokens to send. On ABAP, clients for other vendors' APIs, such as Microsoft AI SDK for SAP for Azure OpenAI and llm_client across chat-completion providers, reach models the SAP library does not.

cap-llm-plugin @geertjanklaps/cds-toon-ai-protocol-adapter @jerome-benoit/sap-ai-provider ui5-cap-ai-features @saptarishi/cds-plugin-llm @shahid.la/cds-db-nlquery-mcp

Licence1

LicenceSAP Developer License Agreement 3.2 CAP§ 3AlwaysCovered SAP materials or a covered CAP customer application are used as training data or to develop or refine an AI model or system.

§ 3 Artificial Intelligence Training. The agreement prohibits use of the covered materials and customer applications for training artificial-intelligence models or systems.

Calling a model for inference is not automatically AI training. Section 3 matters when the covered material or application participates in training, model development, refinement or the text-and-data-mining uses it names.

SAP Developer License Agreement 3.2 CAP →

SourceSAP Developer License Agreement 3.2 CAPchecked 2026-09-16

Artifacts14

14 of 14 artifacts

ArtifactSignals

CDS plugins5

@cap-js/aiValue-help recommendations and SAP AI Core as a CAP service

Signals

Use terms

@geertjanklaps/cds-toon-ai-protocol-adapterProtocol adapter serving CAP data as TOON instead of JSON

Signals

Use terms

@saptarishi/cds-plugin-llmCAP service kind putting eleven model providers behind one API

Signals

Use terms

cap-llm-pluginGenerative AI in CAP with fields anonymised through SAP HANA

Signals

Functional boundary+1

ui5-cap-ai-featuresCDS plugin and UI5 controls for AI text features via SAP AI Core

Signals

Functional boundary+1

MCP servers1

@shahid.la/cds-db-nlquery-mcpMCP server answering plain-English questions against a CDS schema

Signals

none

Packages7

@jerome-benoit/sap-ai-providerVercel AI SDK provider for SAP AI Core

Signals

none

@sap-ai-sdk/ai-apiAI Core control plane API surface of SAP's AI SDK

Signals

none

@sap-ai-sdk/coreShared internals of SAP's AI SDK

Signals

none

@sap-ai-sdk/foundation-modelsClient for foundation models via SAP AI Core and Generative AI Hub

Signals

none

@sap-ai-sdk/orchestrationOrchestration Service client for templating, grounding and masking

Signals

none

@sap-ai-sdk/prompt-registryClient for prompts kept in SAP's prompt registry

Signals

none

cap-java/cds-aiBridge from CAP Java to SAP AI Core, with Fiori field recommendations

Signals

none

Services1

SAP AI CoreAI runtime on BTP with the Generative AI Hub

Signals

none

Tasks4Beta

Shown workingCoversCovers, under conditionsAssessed, not knownNot assessed

Change which model a CAP service callsAlso in BTP/BAIP, ABAP3 covers, 1 under conditions, 1 not known

A CAP service that already calls a language model is pointed at a different one — a newer version, a cheaper one, or a model from another vendor — and the handler that calls it is not rewritten.

Covers where the choice of model is made and what has to change with it. Does not cover whether the answers are as good afterwards, which is agent evaluation, nor the prompt that would have to be rewritten for a different model's habits, which is prompt management. Which models a subaccount is entitled to, and what they cost, is a project question and is not assessed here.

Covers@sap-ai-sdk/orchestrationPuts the model in one property: the documentation says to choose the LLM by setting name in the promptTemplating.model configuration, and that the harmonised API behind it lets a different foundation model be used without the client code changing, with a switch shown as an example. An optional version and params sit beside the name for a model's own settings.

Remaining work: Keep that configuration outside the handler if the switch is to be a deployment change rather than a release, and make sure the model named is one the bound resource group has a deployment for. The package is ESM-only, which a CommonJS CAP project has to deal with before any of this.

Open artifact →

CoversSAP AI CoreHolds the models and the deployment the call names, and puts one harmonised API in front of them, which is what makes a change of model configuration rather than a client change. SAP's own BTP AI best-practices page states it in those terms.

Remaining work: Deploy the model to be switched to in a resource group the application's binding can reach, and keep the deployment id or the model name the call carries in configuration rather than in the handler. Nothing here tells the application which model to use.

Open artifact →

Covers@saptarishi/cds-plugin-llmMoves the choice out of SAP AI Core altogether: eleven provider kinds are registered under cds.requires.llm, the client cds.connect.to('llm') returns is the same whichever is configured, and the README states outright that swapping backends is a configuration change rather than a code change. A development profile on a local Ollama model and a production profile on the Generative AI Hub are the same handler.

Remaining work: Give each kind its own credentials in cds.requires or the environment, and accept that the prompt and the tool schemas are the same text sent to a different vendor's model, which the README does not claim behaves the same. The package is independent and not an SAP-supported dependency.

Open artifact →

Covers, under conditionscap-llm-pluginReaches the same harmonised API from a CAP service: the README's feature list names support for SAP AI Core's orchestration service harmonised chat-completion APIs, so a service already calling the plugin can take a different model through that route.

Conditions: The orchestration route is used: the README lists support for the orchestration service's harmonised chat-completion APIs beside its own simplified chat-completion method, and only the harmonised API is what SAP describes as letting the model change without the client code changing The records and the models stay on SAP HANA Cloud and SAP AI Core, which every feature in this README is written against

Remaining work: Read which of the plugin's two chat paths an existing handler uses, because the simplified chat-completion method is not the harmonised one; the README does not say what changes in the handler when the model does. The README also states that the plugin is not an official SAP product and is published as a community contribution.

Open artifact →

Assessed, not known@jerome-benoit/sap-ai-providerReaches the models SAP AI Core hosts through its orchestration API or its foundation models API, behind the interface a Vercel AI SDK application already calls. What the README does not say is what changes when the model does: it names the models available and the two routes, and no sentence states that the switch leaves the calling code alone.

Remaining work: Read how the model is named when the provider is constructed, which this reading did not establish, before relying on the switch being configuration.

Open artifact →

Cut the tokens a CAP service spends sending its data to a model1 shown working

The same rows a CAP service already hands a model are sent in fewer tokens, so the bill and the latency fall without the service code or the model changing.

Covers the shape of what goes over the wire into a model call. Does not cover choosing a cheaper model, which is the outcome assessed beside this one, nor shortening the prompt itself, which is prompt management, nor retrieving fewer rows in the first place, which is retrieval-augmented generation. What a project is billed is its own contract with SAP or a vendor and is not assessed here.

Shown working@geertjanklaps/cds-toon-ai-protocol-adapterChanges the notation and nothing else: the plugin converts between JSON and TOON, Token-Oriented Object Notation, in both directions at the protocol layer, in a REST variant and an OData one, and its README states that this typically reduces token counts while maintaining data integrity. It counts the tokens on both sides and logs the comparison, so the saving on a project's own payloads is measured rather than taken on trust. It calls no model: it sits where the OData and GraphQL adapters sit, and a consumer selects it by content type.

Remaining work: Point whatever assembles the prompt at the adapter's content type, since the saving only happens on calls that go through it, and read the logged counts to find out whether it earned its place on the shapes this project actually sends. Whether a given model reads TOON as reliably as JSON is not addressed by the plugin and was not read here.

Open artifact →

Run a CAP service's model calls with no SAP AI Core binding1 covers, 2 under conditions

A CAP project whose handlers call a language model starts, serves and answers on a laptop or in a pipeline where no SAP AI Core instance is bound — either from a mock, or from a model running locally.

Covers what answers the call when the binding is absent, and what that answer is worth. Does not cover the vector half of a local run, which is retrieval-augmented generation, nor the mocked users and in-memory database the rest of a local CAP run leans on, which is local development. Whether a developer's machine may hold a model or a vendor key at all is an organisation's rule and is not assessed here.

Covers@saptarishi/cds-plugin-llmReaches a real model with nothing bound: Ollama is one of the eleven provider kinds the README lists, registered under cds.requires.llm like the rest, so a development profile answers from a model on the machine and a production profile from the Generative AI Hub without the handler changing. This is the one way read here where a laptop run gets a model's answer rather than a mock's.

Remaining work: Install Ollama and pull a model, and give the kind its credentials or host in cds.requires or the environment. The answers come from whatever local model is pulled, which is not the model production will use, so a run proves the wiring and not the output.

Open artifact →

Covers, under conditions@cap-js/aiAnswers without the service: the README states that a production deployment requires an SAP AI Core service binding and that without one, local development uses a mock implementation for UI smoke tests. The project starts and the Fiori field shows a recommendation with nothing bound.

Conditions: The recommendations are what is being run: the README names the mock for them, not for every call the AICore service can make The mock is for UI smoke tests, in the README's own words, so what comes back is not a model's answer

Remaining work: Nothing to install for the mock, and nothing it can tell you about the answers a real deployment would give. Anything beyond the recommendations, including the AICore service the plugin embeds, was not read here for a no-binding run.

Open artifact →

Covers, under conditionscap-java/cds-aiThe same shape on the Java side: the README states that without an AI Core binding the plugins fall back to mock implementations for local development, so both the AI Core bridge and the recommendations module answer with nothing bound.

Conditions: Local development: the README scopes the fallback to it The answer is a mock implementation's, not a model's capire's plugin list says the Java version of the AI plugin is currently in alpha

Remaining work: Add the Maven dependency and accept that both modules sit at 0.0.2-alpha on Maven Central, with capire calling the Java version alpha. The README says nothing about reaching a real model without a binding.

Open artifact →

Stream a model's answer out of a CAP service1 shown working, 3 covers

A CAP service receives a model's answer in pieces as the model writes it, rather than one payload after the model has finished, so a chat surface can show text appearing instead of a spinner.

Covers getting the chunks out of the model and into the service. Carrying them on to a browser is the application's own endpoint and, where that endpoint is a socket, WebSocket support; rendering them is the UI's. Whether a long answer should be streamed at all, and what it costs in tokens, is not assessed here.

Shown working@sap-ai-sdk/orchestrationStreams over server-sent events: the documentation states that the OrchestrationClient supports streaming responses for chat completion based on that standard, that stream() returns a stream of chunk responses, and that a streaming request can be aborted with the AbortController API. toContentStream() yields the delta text, getToolCalls() reads the tool calls at the end of a stream, and the orchestration service offers its own streaming options beside the model's, such as a maximum number of characters per chunk.

Remaining work: Carry the chunks to the client, which is the application's own endpoint and not in this package, and decide what a half-written answer does to a handler that was going to validate the whole one. The package is ESM-only, which a CommonJS CAP project has to deal with first.

Open artifact →

Covers@sap-ai-sdk/foundation-modelsThe same mechanism one layer down, without the orchestration modules around it: the documentation states that the AzureOpenAiChatClient supports streaming response for chat completion requests based on the server-sent events standard, with the client constructed on a deployment.

Remaining work: Carry the chunks onward, as above, and accept that the templating, masking and filtering the orchestration client applies around a call are not here, so anything of that kind is the application's own code.

Open artifact →

Covers@jerome-benoit/sap-ai-providerKeeps the stream the Vercel AI SDK already expects while the model comes from SAP AI Core: the README lists streaming support among the package's features, as real-time text generation with structured V3 blocks.

Remaining work: Carry the chunks onward through whatever the Vercel AI SDK application already uses for that, and read which of the package's versioned entrypoints the project is on, since the README names AI SDK 5 to 7 behind separate ones.

Open artifact →

Covers@saptarishi/cds-plugin-llmPuts the stream on the client cds.connect.to('llm') returns: stream is one of the four methods of the unified interface the README's opening sentence names, beside chat, embed and batch, and the README describes it as getting tokens as they arrive from the model instead of waiting for the full response. Because the backend is a profile, the same streamed handler runs against the Generative AI Hub or a local model.

Remaining work: Carry the chunks onward, and configure the provider kind and its credentials. Whether a given provider kind streams as well as the others is not stated per kind in what was read.

Open artifact →

Content34

Group by34 of 34 pieces

LLM integration31

CAP in the age of AI #5 - agents wrap-up

YouTube — SAP Developers playlists, DJ Adams, Johannes Vogt, Simon Engel, Stefan Rudi, Daniel Schlachter, part 5 of 7, Video

What an AI infra engineer sees when opening SAP's AI stack (1/3): the model gateway

SAP Community, amishkush, Blog

UI5 AI Features library is now an npm Package

SAP Community, Wouter Lemaire, BlogThe reasoningRunnable code

Version drift

Show 28 more

CAP in the age of AI #3 - from MCP to A2A

YouTube — SAP Developers playlists, DJ Adams, Johannes Vogt, Stefan Rudi, part 3 of 7, Video

Natural Language Queries for SAP CAP via MCP: The LLM Translates, CDS Executes

SAP Community, Shahid, BlogThe reasoningA recipeRunnable code

Banking Sentinel: A Multi-Agent Credit-Risk Copilot on SAP BTP, Built on a Mixed AI Stack

SAP Community, Shahid, BlogThe reasoningA recipeRunnable code

Core changed

Building an LLM Agent in CAP Java with Spring AI

SAP Community, Johannes Vogt, BlogThe reasoningRunnable code

Core changed

Building an LLM Agent using CAP

SAP Community, Johannes Vogt, BlogThe reasoningRunnable code

Core changed

Agentic AI on BTP: a Multi‑Agent App with Pydantic AI

SAP Community, Wouter Lemaire, Blog

Agentic AI on BTP: a Single Agent with LangGraph

SAP Community, Wouter Lemaire, Blog

Enhancing Enterprise Applications: AI-powered User Experience with SAP CAP AI Plugin

SAP Community, rupamaity, BlogThe reasoningA recipeRunnable code

Integrating a Custom AI Model into SAP CAP Using AI Launchpad

SAP Community, egeAksoyek, BlogThe reasoningRunnable code

Core changed

cds-toon-ai-protocol-adapter: a pragmatic way to cut AI token costs using SAP CAP

SAP Community, Geert-Jan Klaps, BlogThe reasoningA recipeRunnable code

UI5 MCP Server Got Extended for UI Integration Cards Generation

SAP Community, daniel_vladinov, Blog

Prompt-Driven Development: Building Full-Stack SAP CAP Apps from User Stories

SAP Community, AnuradhaDhingan, BlogThe reasoning

Streaming LLM Output with CAP and WebSockets

SAP Community, RobinP, BlogThe reasoningA recipeRunnable code

Core changed

How to Reduce Prompt Token Costs Using Toon = Save Money

SAP Community, Yogananda, BlogThe reasoningA recipeRunnable code

Core changed

Generative AI on SAP BTP

SAP Architecture Center, SAP Architecture Center, Reference architectureThe reasoningA reference

Core changed

How to use Cloud SDK for AI in an existing CAP NodeJS Project in CommonJS

SAP Community, Jacky_Liu1, Blog

Lets Talk to the Database - CAP & SQLite Powered by Google Gemini-2.5-Pro

SAP Community, ShivamShuklaSAP, BlogThe reasoning

Elevating Data Privacy in SAP CAP based GenAI Applications with CAP LLM Plugin

SAP Community, AkashAmarendra, BlogThe reasoningRunnable code

An Innovative Language Exchange App developed with SAP Build Apps, CAP, Kyma, and OpenAI

SAP Community, javieryw, BlogThe reasoningRunnable code

Core changed

Automate your Job Postings Using Generative AI (ChatGPT) & SAP Build Process Automation Apps

SAP Community, shivamshukla12, BlogThe reasoning

Creating a meal generator application with SAP Build, Cloud Application Programming and openAI

SAP Community, former_member793421, BlogThe reasoning

Empowering Your Data Analytics with GPT-3 and SAP CAP Integration

SAP Community, Former Member, BlogThe reasoning

A ChatGPT-like chat app built with modern SAP technologies (CAP, SAPUI5) – Part 3/3

SAP Community, Mike Zaschka, part 3 of 3, BlogThe reasoningRunnable code

1 stale part

A ChatGPT-like chat app built with modern SAP technologies (CAP, SAPUI5) – Part 2/3

SAP Community, Mike Zaschka, part 2 of 3, BlogThe reasoningRunnable code

Gone quiet

A ChatGPT-like chat app built with modern SAP technologies (CAP, SAPUI5) – Part 1/3

SAP Community, Mike Zaschka, part 1 of 3, BlogThe reasoningRunnable code

Gone quiet

Agentic CAP: Build MCP servers and enterprise AI agents

SAP TechEd — session catalogue, Daniel Hutzel, Daniel Schlachter, TalkA recipe

AI-enabled administration: Automating log analysis with generative AI hub

SAP TechEd — session catalogue, TalkThe reasoning

Stop guessing, start counting: How generative AI fixed an inventory mess

SAP TechEd — session catalogue, TalkThe reasoning

SAP AI Core7

What an AI infra engineer sees when opening SAP's AI stack (1/3): the model gateway

SAP Community, amishkush, Blog

CAP in the age of AI #3 - from MCP to A2A

YouTube — SAP Developers playlists, DJ Adams, Johannes Vogt, Stefan Rudi, part 3 of 7, Video

Agentic CAP: Build MCP servers and enterprise AI agents

SAP TechEd — session catalogue, Daniel Hutzel, Daniel Schlachter, TalkA recipe

Show 4 more

AI-enabled administration: Automating log analysis with generative AI hub

SAP TechEd — session catalogue, TalkThe reasoning

🟣 Enable Your AI application to Run on Multiple Models to Ensure Resilience and Compliance

YouTube — SAP Developers playlists, SAP Developers, Talk

🟣 Bring AI into Your ABAP Applications with ISLM: Build, Integrate, and Consume AI Natively

YouTube — SAP Developers playlists, SAP Developers, Talk

Enhancing Business Processes with SAP BTP AI in SAP Integration Suite

SAP Community, arunmaarirajha_kv, Blog

@cap-js/ai1

Enhancing Enterprise Applications: AI-powered User Experience with SAP CAP AI Plugin

SAP Community, rupamaity, BlogThe reasoningA recipeRunnable code

cap-llm-plugin1

Elevating Data Privacy in SAP CAP based GenAI Applications with CAP LLM Plugin

SAP Community, AkashAmarendra, BlogThe reasoningRunnable code

@geertjanklaps/cds-toon-ai-protocol-adapter1

cds-toon-ai-protocol-adapter: a pragmatic way to cut AI token costs using SAP CAP

SAP Community, Geert-Jan Klaps, BlogThe reasoningA recipeRunnable code

ui5-cap-ai-features1

UI5 AI Features library is now an npm Package

SAP Community, Wouter Lemaire, BlogThe reasoningRunnable code

Version drift

@sap-ai-sdk/orchestration1

What an AI infra engineer sees when opening SAP's AI stack (1/3): the model gateway

SAP Community, amishkush, Blog

@shahid.la/cds-db-nlquery-mcp1

Natural Language Queries for SAP CAP via MCP: The LLM Translates, CDS Executes

SAP Community, Shahid, BlogThe reasoningA recipeRunnable code

Seen working9

ArtifactSignals

Samples9

cap-agent-ui5-webui-sampleWorked example of the cap-agent-ui5-webui plugin

No license

cap-openaiCAP service actions calling OpenAI chat completions two ways

No license

cap-ui5-gpt-chatChat app on CAP and SAPUI5 against the OpenAI API

Functional boundary

custom-llm-clientLocal chat client for SAP questions with a CAP LangGraph backend

No license

Sample CAP Application with SAP Cloud SDK for AICAP samples calling SAP AI Core through the SAP Cloud SDK for AI

Functional boundary

SAP-samples/btp-cap-genai-ragCAP reference app for GenAI email support with RAG on HANA Cloud

none

SAP-samples/cap-agentic-engineeredFinancial Risk Analyzer built by an agent grounded in SAP MCP servers

none

SAP-samples/codejam-cap-llmCodeJam on a CAP job posting service using SAP AI Core over OData

none

ui5-ai-features-demoProduct-review screen consuming ui5-cap-ai-features on CAP

No license

Details

What changed — and what it makes stale1

@cap-js/ai 1.0.02026-04-28, CAP

The core gained a standalone AI plugin. Before it, reaching SAP AI Core from a CAP service meant calling its API directly and handling auth and deployments yourself; after it, @cap-js/ai provides a standard AICore CAP service with automatic service-binding resolution and per-tenant resource-group and deployment management, plus out-of-the-box SAP RPT-1 recommendations on value-help fields.

Makes stale:

Guidance that describes reaching SAP AI Core from CAP as something you wire up yourself against the raw AI Core API. That was the only approach before the plugin shipped in April 2026.

Sourcecap-js/ai — Release v1.0.0checked 2026-09-08

Coverage

statecore and community, CAP

assessed2026-10-15

relationshipnarrow by design

domainAI & Agents

also inUI5, BTP/BAIP, ABAP

Comparison properties2

These are the questions used to compare what the artifacts do. Values come from their documentation and may apply to a particular ecosystem or runtime. Each recorded value in the comparison opens its statement and sources.

Models it reachesStreams the response

For your agent

Ref: capability:llm-integration
JSON: /capabilities/llm-integration.json
MCP: Connecting an agent