Agent evals

Here the judge is the system. LLM-Benchmark-ABAP-Code-GenerationLLM-Benchmark-ABAP-Code-GenerationcliA benchmark that has a model write ABAP for 180 tasks ten times each, then feeds the SAP system's own syntax check, activation and ABAP Unit results back for up to five correction rounds.independent, abap creates each generated class in a running ABAP instance and lets the syntax check, the activation and ABAP Unit decide whether it holds, then feeds the error back for another round, so what the numbers report is code that ran rather than code that read well.

Showing artifacts, examples and content with confirmed Standard ABAP applicability. View All ABAP