abap_btocs_ollama

B-Tocs ABAP Plugin - OLLAMA Connector

Summary

This plugin is based on the B-Tocs ABAP SDK and enables the SAP ABAP Server to use Large Language Models provided by the OLLAMA infrastructure services. Most of the Large Language Models can be tested from the ABAP server. For more information about the models and their licences see the OLLAMA Library or the referenced Hugging Face model page.

The plugin is free at your own risk. MIT License.

Architecture

Screenshots

LLAMA2 prompt

LLAMA2 result

MIXTRAL prompt

MIXTRAL result

Server Load

Installation & Configuration

1. Prepare the AI backend

1. Configure the OLLAMA docker container
   1. locally with Docker Desktop - see doc
   2. container stack within B-Tocs container farm - see doc
   3. in an existing docker host or kubernetes cluster - install and configure the OLLAMA container as described here
2. Load your required models, e.g. "llama2", "mistral", "mixtral", ...
3. Test the models from command line within the container
4. check webservice port 11434 is available

2. Prepare your SAP ABAP Backend

1. An installed B-Tocs ABAP SDK is required. Check for Updates.
2. Install this plugin with abapGit.
3. Use Package ZBTOCS_OLLAMA or $BTOCS_OLLAMA (local only)
4. Configure SM59 RFC Destination to your service
5. Test the connection
6. Execute program ZBTOCS_OLLAMA_GUI_RWS_DEMO for a demo

3. Use the OLLAMA Connector

1. Try model 'llama2' with a simple prompt
2. Try the system prompt feature
3. Try other models with same prompts

Example ABAP Code:

DATA lo_response TYPE REF TO zif_btocs_rws_response.
DATA(ls_result) = VALUE zbtocs_ollama_s_generate_res( ).
DATA(lo_connector) = zcl_btocs_ollama_connector=>create( ).

IF lo_connector->set_endpoint(
    iv_rfc     = p_rfc
    iv_profile = p_prf
  ) EQ abap_true.

    lo_response = lo_connector->api_generate(
      EXPORTING
        is_params = VALUE zbtocs_ollama_s_generate_par(
          model       = 'llama2'
          role        = 'user'
          prompt      = 'Wie ist SAP zu einem Weltmarktführer für ERP Systeme geworden?'
          sys_prompt  = 'Antworte in Deutsch und in max. 5 Stichpunkten.'
          template    = ||
          context     = ||
        )
        iv_parse = abap_true
      IMPORTING
        es_result = ls_result
    ).

    IF ls_result IS NOT INITIAL.
      cl_demo_output=>write_text( text = |Response: { ls_result-response }| ).
      cl_demo_output=>display( ).
    ENDIF.
ENDIF.

lo_connector->destroy( ).

Check report ZBTOCS_OLLAMA_GUI_RWS_DEMO for more.

API implementation

The Ollama API is documented here. Not all these API endpoints are implemented in ABAP.

History

• 0.2.0 2024/02/20
  • api endpoint /api/tags
  • api endpoint /api/show
  • api endpoint /api/embeddings
  • added support for images in /api/generate
  • additional information in result popup for /api/generate
  • connector refactoring: parse* methods
• 0.1.0 2024/01/22
  • initial release
  • connector
  • api endpoint /api/generate

Known Issues

1. Some Ollama API features are not implemented yet
2. The performance of the webservice depends on your AI backend. In local environment the performance is bad...
3. There is currently no data center experience for optimal conditions for the container and parallel requests
4. Known issues with backward compatibility. Will be fixed soon...

FAQ

I want to use it as a SAP customer

This plugin is free to use at your own risk (MIT licence).

I want to use it as a SAP partner

It is free for you too. But sharing of ideas and code is expected: Contribute code, docs, ideas, ... to the project via pull requests, blogs, ...

I need support

Support for individual issues is not available. Ask a good developer, partner or the community.

I want to contribute

Fork this repository and contribute through pull requests. More extensive participation in this repository is possible after positive experiences.