Transactional processing of asynchronous and periodic events, with instant dispatch over RedisRedisoutside the atlasAn in-memory data store. For the things here it is the shared cache, the queue or the pub/sub channel that lets several application instances behave as one. Pub/Sub and load spread across every application instance.
Async processingCAPAsync processingcapabilityDoing work outside the request that triggered it, with a guarantee that it eventually happens — deferred jobs, retries, and periodic work. Distinct from messaging, which is about telling other systems.Job schedulingCAPJob schedulingcapabilityRunning something later, or on a cadence, without keeping a process alive to do it — a nightly job, a retry in an hour, a report on the first of the month — and knowing afterwards whether it ran.
RedisRedisoutside the atlasCarries events between application instances over publish/subscribe — the part that makes the queue load-balanced across a scaled app rather than per instance.
Who is behind it
cap-js-communitycap-js-communityOrganisationMaintains this In the atlas: maintains 10@cap-js-communityOliver KlemenzOliver KlemenzPersonMaintains this In the atlas: maintains 4, contributes to 6@oklemenz2maintainersChristian NeuPersonContributes to this In the atlas: contributes to 5@christianneuDr. David A. KunzPersonContributes to this In the atlas: contributes to 16@David-KunzLothar BenderPersonContributes to this In the atlas: contributes to 9@LotharBenderEmanuel BahrPersonContributes to this In the atlas: contributes to 3@PhantomkeksPierre FritschPersonContributes to this In the atlas: contributes to 5@PierreFritschsjvansPersonContributes to this In the atlas: contributes to 45@sjvansMax GruenfelderPersonContributes to this In the atlas: contributes to 7@soccermaxTux240PersonContributes to this In the atlas: contributes to 1@Tux2408 contributors
Before choosing this
crawled
Conditions
LicenceSAP Developer License Agreement 3.2 CAP§ 1, 2, 3A cds-plugin is used with SAP CAP materials, or an artifact is explicitly identified as a CAP plugin or application.
Derived: the instrument's applicability matches this artifact, checked
§ 1License. Productive customer applications are limited to SAP BTP or another SAP-licensed platform, and mass extraction to a non-SAP product is restricted except where solely required for SAP interoperability.
§ 2Intellectual Property. The agreement retains SAP's rights in the covered materials and limits making APIs, tools or software available to third parties, while preserving ownership of customer applications subject to SAP's underlying rights.
§ 3Artificial Intelligence Training. The agreement prohibits use of the covered materials and customer applications for training artificial-intelligence models or systems.
Also, from Async processing§ 1: Messaging is not automatically mass extraction, but an external broker can cross the boundary named in section 1. The data flow, volume and interoperability purpose therefore need to be checked for the concrete design.
PracticalFunctional boundary
If you need sub-second fan-out to external consumers this is the wrong layer — reach for Event MeshEvent MeshserviceManaged queues and topics over AMQP, MQTT and a REST gateway, and the bus SAP's own applications publish their business events to.sap official, btp or Kafka. Without RedisRedisoutside the atlasAn in-memory data store. For the things here it is the shared cache, the queue or the pub/sub channel that lets several application instances behave as one., periodic processing still works but instant dispatch degrades to polling.
What it declares
Every artifact
Type
CDS plugin
Origin
By a community org
Runtime
Node.js
declared
Runs on
Cloud FoundryKyma
declared
As a CDS plugin
Multitenancy
safe
declared
Distributed through
does not say
declared, How the plugin is installed into a CAP project, as its documentation says.
Connections5
Group by
NodeRelationConditionBasis
Functional3
cds-kafkacds-kafkacds-pluginKafka adapter exposing topics as a CDS messaging serviceindependent, cap
→alternative to
none
authored
alternative to cds-kafka
Functional, authored
Kafka adapter exposing topics as a CDS messaging service
Async processingAsync processingcapabilityDoing work outside the request that triggered it, with retriesSAP and community, narrow by design
→provides
none
authored
provides Async processing
Functional, authored
Doing work outside the request that triggered it, with retries
Job schedulingJob schedulingcapabilityRunning something later or on a cadence, and knowing it ranSAP only
→provides
none
authored
provides Job scheduling
Functional, authored
Running something later or on a cadence, and knowing it ran
Technical2
Redis on SAP BTP, hyperscaler optionRedis on SAP BTP, hyperscaler optionserviceManaged Redis from the hyperscaler via a BTP service bindingsap official, btp
→available via
through Redis
available via Redis on SAP BTP, hyperscaler option
Technical, derived, through Redis
derived
available via Redis on SAP BTP, hyperscaler option
Technical, derived, through Redis
Managed Redis from the hyperscaler via a BTP service binding
Carries events between application instances over publish/subscribe — the part that makes the queue load-balanced across a scaled app rather than per instance.
needs Redis
Technical, stated
For
Carries events between application instances over publish/subscribe — the part that makes the queue load-balanced across a scaled app rather than per instance.
Basis
“Event-queue utilizes Redis' publish/subscribe mechanism to distribute events to different application instances.”event-queue — Load balancing, checked
stated
needs Redis
Technical, stated
For
Carries events between application instances over publish/subscribe — the part that makes the queue load-balanced across a scaled app rather than per instance.
Basis
“Event-queue utilizes Redis' publish/subscribe mechanism to distribute events to different application instances.”event-queue — Load balancing, checked
An in-memory data store. For the things here it is the shared cache, the queue or the pub/sub channel that lets several application instances behave as one.
Redisoutside the atlasAn in-memory data store. For the things here it is the shared cache, the queue or the pub/sub channel that lets several application instances behave as one.
@cap-js-community/event-queuecds-pluginTransactional processing of asynchronous and periodic events, with instant dispatch over Redis Pub/Sub and load spread across every application instance.community org, cap
@cap-js-community/event-queue → RedisDocumented onlyCarries events between application instances over publish/subscribe — the part that makes the queue load-balanced across a scaled app rather than per instance.
The Event-Queue is a framework built on top of CAP Node.js, designed specifically for efficient and
streamlined asynchronous event processing. With a focus on load balancing, this package ensures optimal
event distribution across all available application instances. By providing managed transactions similar to CAP
handlers, the Event-Queue framework simplifies event processing, enhancing the overall performance of your application.
Additionally, Event-Queue provides support for periodic events,
allowing for processing at defined intervals. This feature further extends its capabilities in load balancing and
transaction management, ensuring that even regularly occurring tasks are handled efficiently and effectively without
overloading any single instance. This makes it an ideal solution for applications needing consistent, reliable event processing.
Getting Started
Dive right in and check out the getting started documentation.
This project is open to feature requests/suggestions, bug reports etc.
via GitHub issues. Contribution and feedback are encouraged
and always welcome. For more information about how to contribute, the project structure, as well as additional
contribution information, see our Contribution Guidelines.
Code of Conduct
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for
everyone. By participating in this project, you agree to abide by its Code of Conduct at all
times.
Licensing
Copyright 2023 SAP SE or an SAP affiliate company and @cap-js-community/event-queue contributors. Please see
our LICENSE for copyright and license information. Detailed information including third-party components and
their licensing/copyright information is
available via the REUSE tool.
Saga follow-ups (#succeeded, #failed, #done) are now reliably assigned to the transaction of the event which created them when events are processed in parallel. This prevents missing follow-ups and duplicate key errors.
Improved follow-up handling for transaction mode alwaysCommit and alwaysRollback, so that all follow-ups of a chunk with multiple events are inserted.
Improved error handling in transaction mode isolated: a failing event transaction now only sets its own events to error and triggers #failed for them. Other events of the chunk keep their status.
With insertEventsBeforeCommit: false, #succeeded and #done follow-ups are now also inserted if the business transaction is rolled back.
Without Redis, scheduled event processing (e.g. the next tick of a periodic event) no longer nests into the context of the previous run. The growing context chain could cause Maximum call stack size exceeded on long running instances.
Without Redis the scheduled runner (single tenant) run was not guarded against errors. A failing database call — a pool acquire timeout for example — rejected a promise which nobody awaited, so Node reported an unhandled rejection and CAP shut the server down. The run is now guarded like all other runner modes, and the scheduler logs a failing run instead of discarding its promise.
The keep-alive renewed a different lock key than the one acquired for processing, so the lock of an event type was never extended and expired after keepAliveMaxInProgressTime. Another instance could then process the same type and subtype in parallel, and every renew left an orphan row in sap.eventqueue.Lock. Present since v2.0.0.
Broadcasting an event via Redis built the lock key with the namespace twice and therefore never detected a running processing run, so the broadcast was not skipped while the event type was already being processed.
The lock list of the admin service reported namespace and tenant swapped, which made releasing a lock from that list address a key which does not exist.
On PostgreSQL an existing lock in the database could never be taken over. The failed INSERT aborted the transaction, so the takeover in the same transaction failed with current transaction is aborted. The INSERT and the takeover now run in separate transactions.
executeInNewTransaction reused an already open transaction of the caller on every database except HANA. On PostgreSQL this silently turned tx.rollback() into a no-op, so transactions which must be rolled back were committed instead. Only SQLite reuses the transaction now, because it allows a single open transaction only.
Changed
Distributed locks carry an ownership token: renew and release only affect the lock of the processing run which acquired it. A run which lost its lock stops processing further chunks.
Adjustments for CDS 10: upgraded to @sap/cds 10 and @cap-js/db-service, @cap-js/hana, and @cap-js/sqlite 3, and disabled the CDS scheduling service (cds.requires.scheduling: false) as the event-queue provides its own periodic event scheduling. Remaining runtime and dev dependencies were refreshed to their latest versions.
Fixed
Periodic events could get permanently stuck after a long downtime (~30 days or more). The runner only selects events whose createdAt or startAfter is within the last 30 days, but checkAndInsertPeriodicEvents still treated the stale Open occurrence as valid and never re-inserted it, so the event was neither processed nor rescheduled. Stale periodic events that fall outside this window are now refreshed with a new occurrence on startup so they resume processing.
New tenant filter callback tenantIdFilterEventProcessingGlobal for globally excluding tenants from event processing. Unlike the instance-specific tenantIdFilterEventProcessing (which intentionally skips the master-runner lock for sticky-tenant setups, causing every instance to act as a master runner), the global filter returns the same answer on every instance and therefore keeps the master-runner lock. This lets you exclude tenants from processing entirely without increasing the number of open database connections. Both callbacks can be combined; a tenant is processed only if both return true. See documentation.
Periodic cron events could be executed twice if a processing run picked up the event slightly early (within the 3-second start headroom). The next occurrence was calculated based on the current time, which was still before the occurrence that had just been processed, so the same occurrence was scheduled and executed again. The calculation is now based on the planned start time of the processed event if it was picked up early.
The in-memory scheduling of the next cron occurrence is now based on the actual time until the next occurrence instead of always being armed. This avoids redundant long-running timers for infrequent cron events and prevents a setTimeout overflow (which triggered a spurious immediate processing run) for cron cadences greater than ~24.8 days.
[CAP Queue] Support cds.env.requires[<SERVICE>].outboxed as an additional configuration alias alongside outbox and queued. Properties from outboxed override outbox (and are themselves overridden by queued), enabling per-service configuration when a service is registered via the outboxed key.
New initialize option authCacheExpiryReductionMaxPercent (also configurable via cds.env.eventQueue.authCacheExpiryReductionMaxPercent). Shortens the cached XSUAA token TTL by a random percentage in [0, value] per token fetch, providing a safety margin against using nearly-expired tokens and avoiding synchronized expiry across tenants that would otherwise stampede the XSUAA token endpoint. Default 10; allowed range [0, 80] (values outside this range throw on initialization).
[CAP Queue] Add support for event chaining: register #succeeded and #failed successor handlers that are automatically triggered based on the outcome of an event handler. See documentation.
[CAP Queue] Add #done unconditional successor — fires after every event regardless of success or failure, analogous to a finally block. Useful for cleanup that must always run (releasing locks, audit events, etc.).
[CAP Queue] Successor handlers (#succeeded, #failed, #done) can be configured independently in the events section of the service's queued configuration, including event-specific variants (e.g. orderCreated/#succeeded).
[Telemetry] Add opt-in event queue metrics (collectEventQueueMetrics). When enabled together with Redis and an OpenTelemetry MeterProvider, the module publishes cap.event_queue.jobs.pending, cap.event_queue.jobs.in_progress, and cap.event_queue.stats.refresh_age as Observable Gauges, broken down by namespace. See documentation.
[CAP Queue] Allow to propagate cds.context properties (e.g. features). This can be configured per event (cds.env.requires[<SERVICE>].queued.propagateContextProperties = ["features"])
Fixed
The event property retryOpenAfter now works as expected.
[CAP Queue] req.query is not correctly serialized if a cds context with a more complex query is queued/outboxed
Removed redisNamespace: The redisNamespace parameter has been replaced by the new namespace concept. Use the namespace parameter to achieve the same functionality.
Legacy Event Processor behavior change: Events without a returned status are now treated as successfully processed (instead of erroneous). This prevents multiple processing attempts for the same event.
Added
Namespace support for multi-service setups: Introduced namespaces to support scenarios where multiple microservices share the same database (HDI Container). See documentation.
Enhanced CAP outbox service return values: CAP outbox services can now return more detailed information. In addition to the event status, you can now update the startAfter and error fields of an event. See documentation.
Configurable reprocessing for 'Open' event status: With the event configuration property When an event status ofOpen is returned, you can now configure when such events should be reprocessed. This allows you to mark event processing as invalid without increasing the attempt counter.
CDS namespace support in service names: Service names now support CDS namespaces, e.g., cds.env.requires["cds.xt.DeploymentService"].
Support for cds.queued: Added support for the cds.queued property.
[Admin Service] allow to publish events via Admin Service to all or a defined list of tenants
[Event Configuration] propagateHeaders allows forwarding headers from the original CDS context to the outbox call by specifying their names in an array.
[Event Configuration] randomOffsetPeriodicEvents (global default) + randomOffset: This property allows adding a random offset in seconds to periodic events to
stagger their start times and reduce load spikes on the application server.
Fixed
[CAP outbox] correctly connect to service in combination with redis and service specific configuration
[CAP outbox] the features below bring feature parity to CAP outbox services in comparison to EventQueue Classes.
Starting from v1.10.0 it's recommend to use CAP outbox services instead of EventQueue Classes.
[CAP outbox] periodic events for actions/events in CAP outbox
services Documentation
[CAP outbox] allow to specify specific event settings of every action in the CAP
service Documentation
[CAP outbox] enable event clustering with convince helper
functions Documentation
[Event Configuration] timeBucket: This property allows events of the same type to be grouped and processed in batches.
The value of this property is a cron pattern.
Example: */30 * * * * * — This means all events published within 30 seconds are processed together.
[Event Configuration] AppNames (apps on which an event should be processed) now supports regex
expressions. Example: /srv-backend/i
Changed
[CAP outbox] req.context._eventQueue has moved to req.eventQueue
Introduced the ability to configure
a tenant filter
for event processing, enabling features like sticky tenant
processing and dedicated application instances.
Fixed
Resolved an issue with transaction handling where an event handler attaching an on-succeeded callback would throw an
exception.
Prevented error logs from being generated when event configurations are missing.
Added a suspended status for events. Events in this status won't be processed until changed back to open. Useful for
delaying processing when the event isn't ready.
Added an option to filter tenant lists when checking for open events. Published events are still processed for all
tenants, but periodic events can be filtered.
Distributed events via Redis that are not part of the current configuration to support DWC use cases where the same
app might have different software states.
Changed
replace req.user.authInfo with req.user.tokenInfo to following the standard of CAP Node.js
Fixed
long blocked event-loop could lead to stop processing of events
Redis integration: PX values for keys require integer input. Values are longer automatically rounded.
Added support for defining periodic event schedules using cron expressions, providing more flexible and precise
scheduling options beyond simple intervals in seconds.
See documentation.
triggerEventProcessingRedis is a public api now. This can trigger the processing of multiple events via redis.
The central configuration property publishEventBlockList has been introduced, enabling the option to disable the
publication of the event block list to all application instances. The default value is set to true to prevent
immediate impact on existing functionality.
Fixed
do not initialize the event-queue during cds build/compile steps
Enhanced the configuration options to specify not only the application name but also the instance index where the
event should be processed.
See documentation.
Enable more event publish properties for CDS outboxed service.
See documentation.
Introduced the retryFailedAfter configuration option, allowing you to specify the interval (in milliseconds) after
which failed events should be retried, provided the retry limit has not been exceeded.
Increased test coverage for fetching authInfo with @sap/xssec
Export WorkerQueue for monitoring purposes to provide insights into the running load of the application.
JSDocs: added addEntryToProcessingMap for EventQueueProcessorBase
Enhanced Event Processing: Events will continue to be processed even if the initial processing time is exceeded.
Events are now broadcast, allowing different application instances to pick them up. The existing worker queue is used
to ensure proper load balancing.
Default for runInterval changed from 5 minutes to 25 minutes. This is to reduce the load on the database.
The load of the internal periodic event DELETE_EVENTS is increased to 20 to only process 5 tenants in parallel. This
is to reduce the load on the database.
The max processing time for the event-queue is set to the runInterval.
The cleanupLocksAndEventsForDev parameter allows for the cleanup of all locks and events in progress during server
start. This option is intended for development purposes.
Fixed
Allow to initialize event-queue without config.yml in case of usage as CAP outbox
Changed
The parameter runInterval is checked during init. Only values greater than 10 seconds are allowed.
block the run of periodic events via
config documentation
Add a label to the workerQueue. This will help in understanding which events were throttled in case of throttling.
The isEventQueueActive configuration can now be used to deactivate the runtime processing of all events.
with the function getLastSuccessfulRunTimestamp the timestamp of the last successful run for a periodic event can be
requested. documentation
Added a performance tracer for periodic events. This will log any processing that takes longer than 50ms.
Added type and subtype to all performance traces.
Fixed
Catch exception during Redis channel subscription
Endless running if event status open is returned and checkForNextChunk is activated
Changed
Refactored the deletion process for completed events. The default setting for deleting processed events has now been
updated to a 7-day timeframe.
documentation
the configuration variable isRunnerDeactivated is renamed to isEventQueueActive.
Reduced the log severity to debug for "Selected event queue entries for processing" during periodic events.
cds-kafkacds-kafkacds-pluginA Kafka adapter for CAP that registers Kafka topics as a CDS messaging service over KafkaJS, so emit and on work unchanged, with per-message keys, partitions and topics set through headers.independent, capKafka adapter exposing topics as a CDS messaging service
Async processingAsync processingcapabilityDoing work outside the request that triggered it, with a guarantee that it eventually happens — deferred jobs, retries, and periodic work. Distinct from messaging, which is about telling other systems.SAP and community, narrow by design
ABAP-PRCABAP-PRCpackageA framework for custom multi-step ABAP processes: a process class declares the states, and the framework runs each document through them and monitors the run.independent, abapFramework for multi-step ABAP processes with monitoringfrom ABAP
Event MeshEvent MeshserviceManaged queues and topics over AMQP, MQTT and a REST gateway, and the bus SAP's own applications publish their business events to.sap official, btpManaged queues and topics over AMQP, MQTT and RESTfrom BTP/BAIP
com.sap.cds:cds-services-bomcom.sap.cds:cds-services-bompackageThe Maven bill of materials for CAP Java, pinning every com.sap.cds module in a release, from the API and core runtime to the Spring Boot starters and the CQN engine, to the same version.sap official, capMaven bill of materials pinning CAP Java module versionsfrom CAP
@cap-js-community/sap-afc-sdk@cap-js-community/sap-afc-sdkpackageSDK for building SAP Advanced Financial Closing scheduling providers on CDS.community org, capSDK for SAP Advanced Financial Closing scheduling providers on CDSfrom CAP
cds-event-monitoringcds-event-monitoringcds-pluginA CAP plugin that stores every event received on its configured messaging topics in a table, with a Fiori-annotated service to browse them, resend events and drain a dead-message queue.independent, capStores, browses and resends messages a CAP service receivesfrom CAP
@odatano/watch@odatano/watchcds-pluginWatches Cardano addresses, payment credentials and minting policies from inside CAP, emitting events with full UTxO payloads and persisting them so a missed window can be replayed.independent, capCAP plugin watching Cardano addresses and emitting eventsfrom CAP
@sap/cds@sap/cdspackageThe CAP Node.js runtime, which compiles the CDS model, serves the services declared in it, answers generic requests before any handler runs, and loads every cds-plugin.sap official, capCAP Node.js runtimefrom CAP
cds-natscds-natscds-pluginNATS as a messaging service for the CAP Node.js runtime, the broker behind cds.connect for emit and subscribe, for teams already running NATS rather than Event Mesh or Redis.independent, capNATS messaging service for the CAP Node.js runtimefrom CAP
Job schedulingJob schedulingcapabilityRunning something later, or on a cadence, without keeping a process alive to do it — a nightly job, a retry in an hour, a report on the first of the month — and knowing afterwards whether it ran.SAP only
SAP Job Scheduling serviceSAP Job Scheduling serviceserviceCalls an application's endpoint on a schedule or once at a time and keeps the log of every run — cron without a process of your own to keep alive.sap official, btpCalls an application endpoint on a schedule and logs each runfrom BTP/BAIP
@sap/cds@sap/cdspackageThe CAP Node.js runtime, which compiles the CDS model, serves the services declared in it, answers generic requests before any handler runs, and loads every cds-plugin.sap official, capCAP Node.js runtimefrom CAP
Verdict rings retired. What the ring compressed now lives in readings derived from facts (liveness, compatibility, reach) and in the description fields, so a thing can be dead as a dependency and still worth reading. VISION §9.
