Agent evals

On BTP evaluation sits beside the model: SAP AI Core's generative AI hub compares prompts and models in SAP AI Launchpad before a prompt reaches application code.

Overview

SAP SHIPS

The generative AI hub in SAP AI Core has an Evaluation Service that benchmarks a use case across models with defined metrics against test datasets, a prompt optimizer that adapts prompts to a target model, and a prompt registry that versions the templates.

Not covered: At the prompt and the model. An evaluation runs a registered prompt against a dataset somebody prepares, so fixtures drawn from the application's own data, and a run on every change to the application, remain the project's to build.

SAP's TechEd workshop teched2025-AI161 demonstrates and does not provide; the SAP side rests on the AI Core services as the SAP Architecture Center describes them. How the independent skill collection came to stand beside them is not documented in any page retrieved.

SourceGenAI Applications — SAP Architecture Centerchecked 2026-09-14

THE COMMUNITY ADDS

DanKromp/skillOverflow lists BTP among the stacks it serves and ships an eval with each skill, so it brings the coding-agent kind of eval to this door; the AI Core services above compare prompts and models.