@cap-js-community/mtx-tool

CHANGED

• dependencies: update node-fetch to v3 with require(esm). node v20 can do this by default now: https://nodejs.org/en/blog/release/v20.19.0/#requireesm-is-now-enabled-by-default:#requireesm-is-now-enabled-by-default

ADDED

• svm: new --svm-restart, --svm-restart-skip commands that triggers a zero-downtime rolling restart for all running apps bound to the service-manager instance.

FIXED

• svm: retry-after logic expected the standard header, but svm uses X-RateLimit-Retry-After instead.

We reworked the service-manager interactions. The operations for credential rotation become idempotent now, making outside retries possible.

The old commands still work but are now deprecated: they emit a warning and delegate to the new implementations. Migrate to the replacements below, as the deprecated commands will be removed in a future release.

The full rotation is a five-step choreography of the two idempotent commands:

1. rolling restart — converge apps onto the most-recent binding.
2. --svm-make-bindings-single — prune to one binding per instance.
3. --svm-make-bindings-double — add the new binding.
4. rolling restart — apps switch to the new binding.
5. --svm-make-bindings-single — prune the old binding.

If the fleet is already at one binding per instance, skip to step 3.

CHANGED

• npm: use bundleDependencies instead of shrinkwrap to ship the whatwg-url override to users. shrinkwrap is no longer honored as of npm v12.

• svm: reworked --svm-repair-bindings to --svm-make-bindings-single and --svm-fresh-bindings to --svm-make-bindings-double. both are now idempotent and have the SERVICE_PLAN TENANT_ID calling args, making them consistent with other svm commands.

• svm: --svm-repair-bindings, --svm-fresh-bindings, and --svm-refresh-bindings are deprecated. they still work but now emit a warning. they are reimplemented on top of the new idempotent binding logic in a way that reproduces their previous behavior, and will be removed in a future release. migrate to --svm-make-bindings-single, --svm-make-bindings-double.

FIXED

• reg: subscription listing now uses the cursor-based paging APIs (/subscription-manager/v2/subscriptions and /saas-manager/v2/application/subscriptions) instead of page-number paging, keeping pagination reliable for large result sets (fixes #197).

• core: mtx cache now dedupes concurrent lookups of the same configured app name, avoiding redundant cf requests.

CHANGED

• core: service credentials are now retrieved through the cloud-foundry service_credential_bindings API instead of reading VCAP_SERVICES from the app's environment. this is a meaningful step towards supporting service-keys in addition to app bindings. credential lookups no longer depend on whether VCAP_SERVICES is provided inline or via file.

• srv: --server-env now handles file-based VCAP_SERVICES (fixes #140).

• core: cache window for cf app info was shortened from 12 hours to 4 hours.

• hdi, svm: readonly list commands --hdi-list, --hdi-long-list, --svm-list, --svm-long-list now reuse the persisted app-info cache.

ADDED

• svm: add client headers Client-Name and Client-Version for all calls to service-manager.

FIXED

• cds: list filter for subdomain works again.

• cds: works with xsuaa broker plan again (fixes #161).

ADDED

• svm: new command --svm-fresh-bindings to create new bindings without deleting the current bindings. this is meant for scenarios, where you create new bindings, do rolling restarts, and then run --svm-repair-bindings for cleanup.

• svm: new bindings created via --svm-fresh-bindings, --svm-refresh-bindings, or --svm-repair-bindings will have a managing_client_lib label if their associated instances have the offering hana and plan hdi-shared. this specific label was expected by cap java until recently.

FIXED

• svm: new bindings created via --svm-fresh-bindings or --svm-refresh-bindings will combine the labels from the associated instance and from the previous binding.

⚠️ This release is potentially disruptive. The configured app used for cds commands now needs to be bound to an auth service: both xsuaa (UAA) and identity (IAS) are supported. For most projects, this will be the case already.

CHANGED

• core: require node engine >= 20.0.0

• cds: now supports IAS-only, unless the @sap/cds-mtxs endpoints are protected by a custom mTLS middleware.

FIXED

• core: move from deprecated url.parse() to silly WHATWG API.

CHANGED

• [experimental] core: requests that fail and get a retry, will be logged with a request id [req-xx] for easier correlation.

• [experimental] core: extended list of correlation headers to X-CorrelationId, X-Correlation-Id, X-Request-Id, X-Vcap-Request-Id.

ADDED

• set: added a new configuration option for an app to access subscription manager.

• reg: all reg commands will work with either subscription manager or saas registry or both depending on what is configured with set (fixes #100).

• reg: new command --registry-migrate to migrate one tenant from saas registry to subscription manager. this operation is irreversible and requires that the tenant's subaccount trusts an IAS with origin key sap.custom.

FIXED

• cds: cdsl will correctly display values for tenants held in either saas registry or subscription manager (fixes #127).

CHANGED

• svm: new bindings inherit all labels from the underlying service instance.

• svm: new bindings always get a service_plan_id label, which is expected by @sap/cds-mtxs.

ADDED

• [experimental] set env variable MTX_CORRELATION=1 to log correlation information for service responses. we will show the first matching header of: X-CorrelationId, X-Correlation-Id, X-Vcap-Request-Id.

FIXED

• svm: --svm-repair-bindings and --svm-refresh-bindings will ignore instances with ready: false.

⚠️ This release is potentially disruptive. We carved out the service-manager interactions from hdi into a new section svm. Among other things, this allows us to handle credentials rotation for all managed service plans consistently, not just for HANA containers.

REMOVED

• srv: command --server-start-debugger is not needed anymore, this happens implicitly now with --server-debug.

CHANGED

• hdi: all binding manipulation commands have migrated to dedicated service-manager.

ADDED

• svm: new dedicated command section for service-manager interactions (fixes #87).

• svm: coding that creates or deletes bindings has some built-in retries for temporary outages.

• cds: cds upgrade commands will download tenant deployment logs from cf app container.

FIXED

• svm: --svm-repair-bindings command will now clean up bindings with ready: false.

Added

• cds: offer an optional flag --first-instance for --cds-upgrade-all that will cause all tenants to be processed by the first app instance only. this will make the upgrade less resource efficient, but resilient to auto-scaling. (fixes #104)

Fixed

• hdi: change internal logic to identify the relevant service both by offering name hana and plan name hdi-shared. this takes one more request, but it's much cleaner. before, we had implicitly assumed that no other service offering uses the plan name hdi-shared.

Fixed

• srv: fix debug command type mixture bug where app instance argument was ignored

• srv: fix certificate dump command

Fixed

• cds: fix cds upgrade safety net for stalled upgrades

• the whatwg-url override did not reach users, because it only works locally. this should be fixed by using shrinkwrap.

Removed

• node v16 is no longer supported

• hdi: remove legacy instance manager code

• cds: remove legacy cds-mtx code

• reg: remove registry job command

• srv: remove server info command

Changed

• marked --server-start-debugger as not dangerous

Added

• established baseline test coverage

• hdi: list relations also gets the --json flag

Fixed

• override whatwg-url to v14. this fixes a punycode incompatibility warning when using the commonjs variant of node-fetch with node v21+.

• fix some internal context call queuing under parallel execution

• fix console output for --server-start-debugger and --version

Added

• reg/cds/hdi: the list and long list command gets a --json flag to produce json output (fixes #86)

• uaa: all uaa commands get a --json flag to produce json output

• code scanning with codeQL

Fixed

• added node v22 to voters

• better separation for test request-replay recording and playback

• hdi: fixed logic for ready list column, such that it's false if either instance or binding is not ready

• srv: better resilience for apps with no buildpack

Added

• reg: new --only-stale and --only-failed filter options for most registry commands. stale here means that the last changed on day is older than the invocation day.

Fixed

• reg: --registry-update-all is more resilient for failing calls

• better error message for users with access restrictions due to new btp space supporter role

Fixed

• cds: upgrade now fails with a non-zero error code if an underlying task fails

Changed

• reg: registry service calls now happen concurrently where appropriate similar to hdi and cds interactions.

Added

• added environment variables to control polling frequency for interactions:

  environment variable	effect
  MTX_CDS_FREQUENCY	change polling frequency milliseconds for server async job calls (default is 15000)
  MTX_REG_FREQUENCY	change polling frequency milliseconds for service async job calls (default is 15000)

Fixed

• reg: better error handling for registry updates

Fixed

• reg: fixed a paging bug where most registry commands had an infinite loop if more than 200 tenants are handled. (reported by @cgaillydetaurines)

Changed

• cds: tenant upgrade now logs progress of individual tasks (@sap/cds-mtxs uses one task per tenant) for every poll. the logs will look something like:

  GET https://my-server-mtx.cfapps.sap.hana.ondemand.com/-/cds/jobs/pollJob(ID='d2f560dc-542c-4091-a632-919c941210b4') 200 OK (163ms)
  job d2f560dc-542c-4091-a632-919c941210b4 is RUNNING with tasks queued/running:  5/ 5 | failed/finished:  0/ 0
  GET https://my-server-mtx.cfapps.sap.hana.ondemand.com/-/cds/jobs/pollJob(ID='d2f560dc-542c-4091-a632-919c941210b4') 200 OK (98ms)
  job d2f560dc-542c-4091-a632-919c941210b4 is RUNNING with tasks queued/running:  4/ 5 | failed/finished:  0/ 1
  GET https://my-server-mtx.cfapps.sap.hana.ondemand.com/-/cds/jobs/pollJob(ID='d2f560dc-542c-4091-a632-919c941210b4') 200 OK (98ms)
  job d2f560dc-542c-4091-a632-919c941210b4 is RUNNING with tasks queued/running:  1/ 4 | failed/finished:  0/ 5
  ...
  

• cds: tenant upgrade now tracks that task progress is still happening. if no progress is detected for 30 minutes, it will show the final status for each tenant and fail.

Added

• uaa: some arguments can be passed in via environment variable instead of the commandline. this behavior is expected by Jenkins and Github Actions. for example, the following is now possible:

  UAA_USERNAME=user UAA_PASSWORD=pass mtx --uaa-user
  

Fixed

• sensitive input arguments are now masked in the running console output

Fixed

• removed npm postinstall script

Changed

• hdi: all accesses to service instances and bindings now filter for the service-plan hdi-shared. this change allows the service-manager to be used with other services, e.g. redis-cache, alongside HANA containers.

Fixed

• uaa: uaasc, uaasp, uaasu error handling and message with passcode url was improved.

Changed

• reg: improved regl and --registry-update support for instances with service plan. (proposal and provisioning support by @andre68723)

Removed

• hdi: --hdi-migrate-all command is removed. Instance manager has been deprecated for a while, so migrations to service-manager have happened by now.

Changed

• uaa: uaas --uaa-service command has been renamed to uaasc --uaa-service-client. This fits better with the pattern of new commands.

Added

• uaa: new flexible commands, so that either the regular server or a trusted services can be accessed as a technical client, as a user with passcode, or as a user with username and password. (fixes #62 by @soccermax)

  uaac   --uaa-client [TENANT]                                  obtain uaa token for generic client
  uaap   --uaa-passcode PASSCODE [TENANT]                       obtain uaa token for one-time passcode
  uaau   --uaa-user USERNAME PASSWORD [TENANT]                  obtain uaa token for username password
  uaasc  --uaa-service-client SERVICE [TENANT]                  obtain service token for generic client
  uaasp  --uaa-service-passcode SERVICE PASSCODE [TENANT]       obtain service token for one-time passcode
  uaasu  --uaa-service-user SERVICE USERNAME PASSWORD [TENANT]  obtain service token for username password
  

Fixed

• uaa: fix error with recommended passcode url when credential-type x509 is used. (fixes #59)

Changed

• cds: onboard now supports arbitrary subscription metadata https://cap-js-community.github.io/mtx-tool/cap-multitenancy/#onboard-tenant

Added

• srv: new functionality to dump certificates from app server (pr #53) contributed by @kkoile

Changed

• srv: environment dump now exports VCAP_APPLICATION (pr #55) contributed by @kkoile

Added

• cds: added missing timestamp functionality (fixes #46)

• cds,hdi: allow users to control request concurrency with env variables MTX_CDS_CONCURRENCY, MTX_HDI_CONCURRENCY

Changed

• cds,hdi: changed default request concurrency from 5 to 10

Added

• add response time to all logged requests

• reg: enable access for registry instances with service plan

Fixed

• srv: fix debug command for case where uaa is not configured

Added

• reg: optional flag to skip unchanged dependencies for updates (fixes #38)

Added

• out of the box support for deploy with confidence "uuid-versioned" app suffixes (fixes #35)

Fixed

• cds: auto undeploy for mtxs (#34)

• hdi: allow hdi tenantIds to contain forward slash

• uaa: make uaa service token work with user provided services

Added

• reg: enable update subscribe apps' url (fixes #25) contributed by @jiangxin0503

Fixed

• homogenize single and multi tenant registry calls

• change function signatures to avoid null parameters

• reg: common enum for registry response states

Fixed

• respect uaa-token expiry (fixes #22)

• code cleanup

Fixed

• documented --force arg for scripts (relates to #20)

• reg: --registry-update-all now retries tenants with previously failed updates

Added

• uaa: expose uaa userinfo (fixes #13)

Fixed

• hdi: fix access problems for hdi containers with no associated tenant

• srv: allow server-start-debugger to pass an app instance

Added

• first external release

Fixed

• cds: mtxs cds-upgrade error did not cause a non-zero error code

Added

• cds: support cds-mtxs

Fixed

• better pagination for cf requests

• reg: support enforced pagination in saas-registry api

Added

• hdi: new command --hdi-list-relations to show mapping of service-manager containers to their bindings. makes it easy to see if some containers have too many or no binding, or conversely, if some bindings have no container.

• hdi: new command --hdi-repair-bindings to harmonize service-manager containers and bindings, so they are 1-to-1

Fixed

• hdi: add service manager instance information to hdi long list

• hdi: allow --hdi-rebind-* to create new bindings with custom parameters

Fixed

• hdi: rebind all tenants with one command

• hdi: hdi list instances and bindings separately

Changed

• rollback required engine to node v14 npm v6.14. Note: support for node v14 will end shortly after the official end of life 2023-04-30.

Added

• upgrade required engines to node v16 and npm v8

Fixed

• cds: fixes cds-upgrade-all bug where number of tenants is below number of app instances

• cds: update docs with tenant upgrade scaling info

Added

• cds: cds-upgrade-all is now spreads the load across app instances

Fixed

• rework internal error handling

• reorganize some internal reuse functions

Fixed

• hana cloud cannot tunnel warning for combination of hditt and port 443

Removed

• various dedicated paas commands for reg/cds/hdi areas have been removed. provider subaccount should not be treated specially here.

Changed

• uaa: combined logically similar commands:

  • uaap and uaas become uaac,
  • uaac and uaasc become uaap

• srv: srvda app_instance_id argument is now optional and defaults to zero

Fixed

• regl/regll, cdsl/cdsll, hdil/hdill all allow filters

• uaa: uaac can handle both subdomain and tenant id

• updates for tests and documentation related to new command line interfaces

Fixed

• use cf oauth-token instead of cf curl for better testability

• testing reg/cds/hdi with nock and anonymize playback files

• better status reporting for failed requests

• reg: make registry tenant update serial

• uaa: allow uaa code to handle tenantId

Added

• rename repo to btp-tools-mtx

Added

• cds: switch to manual control of auto undeploy

Fixed

• cds: handle some tenant upgrade errors better

Fixed

• uaa: allow 32-char passcode length

Fixed

• hdi: bugfix in hditp / hditt where tunnel is not opened correctly

Fixed

• hdi: work properly with multiple service-bindings to a single (hdi) service-instance

• resilience with respect to missing cfRouteDomain

Added

• support for x509 enabled uaa bindings

• allow readonly commands for apps with -live suffix

Added

• hdi: separate hdi tunnel information into runtime and design time user

Fixed

• better error handling in cf curl

• hdi: missing await for delete hdi tenant with service manager

Fixed

• change order in cf target log to api/org/space

• cds: write cds upgrade buildlog

• hdi: expect service-manager first

• hdi: change order in hdil so that tenant_id is leading and gets sorted

• hdi: enhance hdil with host information for multi-db environments

• srv: bugfix for --server-debug-app

Added

• support for blue-green projects

Added

• switch to minimal engine node 12 for eslint 8

• hdi: new command --hdi-migrate-all to migrate hdi containers from instance-manager to service-manager

Fixed

• hdi: fix tablelist in hdit*

Added

• reg: new registry-update-all command

• srv: new server-debug-app command

• allow --force flag to override danger guard

Fixed

• reg: add global account id to regl

• tablelist with row numbers by default

Added

• add cds-onboard-tenant

Fixed

• proper handling when err.message is undefined

Added

• server start debugger feature

Fixed

• uaa tests

• tablelist handles sorting on columns where values are undefined

Added

• allow uaa service-plan broker

• registry-list and hdi-list with optional timestamps (new --time flag)

Fixed

• show cf target with org/space/api before danger-guard

• more user input validation for pass arguments and flag arguments

• extend tests with pass argument consistency validation

Fixed

• hdi: can now rebind hdi-container for credential rotation

Added

• documentation for service manager support

Fixed

• reg: more relevant information extracted for regl

• bugfix for uaad

Added

• hdi: service-manager support

Fixed

• uaa: more resilient way to get service tokens

Added

• moved repo to cds-community

Added

• uaa: added saas service token feature

Fixed

• hdi: automatically use next free sid port for hdi tunnel

Added

• reg: added registry update dependencies

Fixed

• reg: rewrote registry offboard subscription for skipping multiple apps

• spawned subprocesses get cleaned up on SIGINT/SIGTERM

Fixed

• better handling of the case when an app has no route

• better text for hidden passwords

Fixed

• fix regression for buildpack location in CF V3 API

Added

• switch to CAPI v3 https://v3-apidocs.cloudfoundry.org/version/3.91.0/

Fixed

• add app-version check to cache

Fixed

• cds: second pass on error-handling for cds-upgrade

Fixed

• mark long-list commands as uncommon

• cds: fix error-handling during cds upgrade polling

Added

• add flagArgs like --decode (for uaa) and --reveal (for hdi)

Fixed

• passwords are now confidential by default

• decrease cache gap to 12 hours to enable a quicker discovery for changes

• show partial outputs for sub-exec failures

Fixed

• cds: more resilient upgrade-tenant

Fixed

• cds: failing upgrade-tenant should cause a non-zero error code

Added

• srv: add command to extract env information

Fixed

• cds: increase polling time to 20sec

• cds: fix instance for upgrades and use capitalized headers consistently

Added

• srv: add app-instance-index to openTunnel

Fixed

• srv: get debugPort through /info and add instance information

• srv: better handling of dynamic buildpacks from Github

• srv: better handling for openTunnel without passing appInstanceIndex

Added

• initial release