@odatano/nightgate
Indexes the Midnight blockchain and submits transactions to it from CAP, exposed as OData V4, with command-line tools for schema deltas, key rotation and migration.
Facts
Where it sits
Who is behind it
In the atlas: maintains 4@ODATANOmaintainerMaxContributes to this
In the atlas: contributes to 4@maxalexweber11 contributor
Before choosing this
Conditions
LicenceSAP Developer License Agreement 3.2 CAP§ 1, 2, 3A cds-plugin is used with SAP CAP materials, or an artifact is explicitly identified as a CAP plugin or application.
- § 1 License. Productive customer applications are limited to SAP BTP or another SAP-licensed platform, and mass extraction to a non-SAP product is restricted except where solely required for SAP interoperability.
- § 2 Intellectual Property. The agreement retains SAP's rights in the covered materials and limits making APIs, tools or software available to third parties, while preserving ownership of customer applications subject to SAP's underlying rights.
- § 3 Artificial Intelligence Training. The agreement prohibits use of the covered materials and customer applications for training artificial-intelligence models or systems.
What it declares
Every artifact
- Type
- CDS plugin
- Origin
- Independently maintained
As a CDS plugin
- Declared for
- >=10 <11
- Distributed through
- npm
Connections
Exposing data and operations through OData
MCP server exposing the NIGHTGATE Midnight attestation layer
Download
Install
npm add @odatano/nightgateDerived from the package name. No recipe recorded; the README has the configuration.
README
quoted from ODATANO/NIGHTGATE under Apache-2.0, readNIGHTGATE - OData for Midnight @odatano/nightgate

SAP CAP plugin: Midnight blockchain indexer + transaction submission, exposed as OData V4.
@odatano/nightgate connects SAP CAP to the Midnight blockchain. A crawler indexes blocks from a Substrate RPC node into CAP entities; a wallet stack in a worker thread submits transactions (Compact deploys and calls, NIGHT and custom-token transfers, dust generation, fee sponsoring). Everything is exposed as OData V4.
┌──────────────────────────────────────┐
│ Midnight Preview / Preprod │
│ Substrate Node GraphQL Indexer │
└──────────────┬──────────────┬────────┘
│ │
wss:// │ Substrate │ GraphQL
JSON-RPC │ RPC │ HTTP + WS
▼ ▼
┌───────────────────────────────────────────────────────────────────────────┐
│ NIGHTGATE │
│ │
│ Main thread Worker thread │
│ ┌──────────────────────┐ ┌──────────────────────────┐ │
│ │ Crawler │ │ Wallet SDK │ │
│ │ - BlockProcessor │ │ │ │
│ │ - reorg detection │ │ - facade.start (sync) │ │
│ └─────────┬────────────┘ │ - transferTransaction │ │
│ │ atomic writes │ - finalize (ZK prove) │ │
│ ▼ │ - registerForDustGen │ │
│ ┌──────────────────────┐ │ - deployContract │ │
│ │ CAP DB │◄────state-save──┤ - submitContractCall │ │
│ │ (SQLite / HANA) │ periodic save │ │ │
│ └─────────┬────────────┘ │ - private-state-rpc │ │
│ │ OData V4 └──────────┬───────────────┘ │
│ ▼ │ │
│ 5 services on /odata/v4/midnight{,-indexer,-analytics,-admin,-verify} │
└───────────────────────────────────────────────────────────────────────────┘
The wallet SDK runs in a worker thread because its sync saturates the microtask queue; the CAP request pipeline stays responsive.
Quick start
npm ci
npm run dev
Or standalone with Docker (docs/docker.md):
docker pull ghcr.io/odatano/nightgate:latest
docker run -d -p 4004:4004 \
-e ENCRYPTION_KEY=$(openssl rand -hex 32) \
-e NIGHTGATE_HTTP_PASSWORD=change-me \
-v nightgate-data:/data \
ghcr.io/odatano/nightgate:latest
.env (see .env.example):
NIGHTGATE_NETWORK=preprod
NIGHTGATE_NODE_URL=wss://rpc.preprod.midnight.network/
# GraphQL indexer, HTTP; the WS URL is derived
NIGHTGATE_INDEXER_HTTP_URL=https://indexer.preprod.midnight.network/api/v4/graphql
# Unset = in-process wasm proving. Setting it selects server proving (production).
# NIGHTGATE_PROOF_SERVER_URL=http://localhost:6300
NIGHTGATE_CRAWLER_ENABLED=false
ENCRYPTION_KEY=<random secret>
First sync walkthrough: docs/quickstart.md.
Services & capabilities
| Service | Path | Content |
|---|---|---|
NightgateService | /odata/v4/midnight | chain data, wallet sessions, token / contract / attestation actions |
NightgateIndexerService | /odata/v4/midnight-indexer | sync state, health, metrics, crawler control |
NightgateAnalyticsService | /odata/v4/midnight-analytics | aggregate counts |
NightgateAdminService | /odata/v4/midnight-admin | sessions, contract registration, diagnostics |
NightgateVerifyService | /odata/v4/midnight-verify | unauthenticated state verification (NIGHTGATE_PUBLIC_VERIFY=true) |
Submit actions are async: they return { jobId, status }; poll getJobStatus(jobId, sessionId). Signatures, error codes and examples: docs/actions.md.
| Capability | Surface |
|---|---|
| Block indexing | Crawler with reorg detection; OData queries on Blocks, Transactions, ContractActions, UnshieldedUtxos, NightBalances |
| Wallet sessions | connectWallet (viewing key, read-only), connectWalletForSigning (BIP39 mnemonic, Lace-compatible HD derivation); AES-256-GCM at rest, bound to the requesting user |
| Token ops | sendNight (ledger from the receiver address; tokenTypeHex for custom tokens), registerForDustGeneration / deregisterFromDustGeneration |
| Fee sponsoring | Dust generation delegation (dustReceiverAddress) and per-tx sponsorSessionId on submit actions (platform sponsors: NIGHTGATE_FEE_SPONSOR_SESSION) |
| Pre-flight | getWalletBalance, estimateSendNightFee, deriveWalletInfo |
| Compact contracts | deployContract, submitContractCall, submitContractCallBatch on registered artifacts |
| Proving | wasm in-process (default without a proof server) or server (selected by a proof-server URL; production). Override: NIGHTGATE_PROVING_MODE |
| Document anchoring | anchorDocument / verifyDocument: hash on chain, storage stays with the caller |
| Document ingestion | prepareDocumentProof: canonical JSON -> payloadHash + salted content root with per-field inclusion paths (16 slots, 32 on attestation-vault-32); prepareMembershipSet: canonical allow-list root. Compute-only |
| Agent access | createAgentGrant / createAgentGrants / updateAgentGrant / rotateAgentGrantToken / revokeAgentGrant: scoped bearer tokens (x-agent-token) with action allow-list, budgets, pinned session and sponsor; attestAgentOutput: verifiable agent-output provenance. MCP server: @odatano/nightgate-mcp |
| Agent markets | sponsorSwap settles a shielded swap from two proven halves; the offer board (postSwapOffer / listSwapOffers / getSwapOffer / retireSwapOffer, anonymous getBoardStatus) carries maker halves as offer files; mintFactoryToken mints a session's own named tokens on the token-factory lineage |
| Field proofs | issueFieldPredicateAttestation (value <= / >= threshold), issueFieldEqualityAttestation (value behind a public digest), issueFieldMembershipAttestation (one of up to 64 allowed values); issueFieldPredicateAttestationBatch: up to 8 mixed claims in one tx. Values stay hidden |
| Cross-document proofs | issueDocumentIntegrityAttestation (B differs from A only in a slot mask), issueDocumentDiffAttestation (at least k slots differ); same width only |
| Crawler-free verification | verifyAttestationState, verifyPredicateState, reindexDisclosures: live contract state from the public indexer, optional network override |
| Tiered disclosure | grantDisclosure / revokeDisclosure / registerGranteeIdentity, DisclosureGrants index, AttestationService with three tiers |
| Local tx building | @odatano/nightgate/txbuilder or @odatano/nightgate-tx: build, prove and sign with your own key, then sponsorFinalizedTransaction / sponsorUnboundTransaction pays and submits. The sponsor sees no key, witness or preimage |
| Browser | @odatano/nightgate/browser + GET /zk-config/<contract>/… + GET /contract-manifest: wallet-driven dApps without Compact toolchain or managed/ artifacts |
| Operations | Health, liveness, readiness, Prometheus metrics, crawler pause / resume / reindex, offline start |
Documentation
- Quickstart: first wallet-signed transaction
- First anchor on the hosted API: key, seed and a shared sandbox vault, no indexer; AGENT_MS is a working agent built that way
- Actions: every action and function with examples
- Architecture: worker thread, submission flow, persistence
- Operations: scripts, local indexer, troubleshooting
- Docker: standalone container, configuration, schema upgrades
- Transaction builder: build sponsorable transactions without a server
- Reference: configuration and project structure
- Changelog
Use as a CAP plugin in another app
cd my-cap-app
npm install @odatano/nightgate @cap-js/sqlite
{
"cds": {
"requires": {
"db": { "kind": "sqlite" },
"nightgate": { "network": "preprod" }
}
}
}
Then cds watch. network is the only required key; the defaults are the public RPC and indexer and wasm proving. Configuration: docs/reference.md#configuration.
Development
npm run dev # cds watch, 12 GB heap
npm run serve:sync # cds-serve, 12 GB heap; for long syncs and e2e runs
npm run sync:start # create a wallet session on the running server
npm run typecheck
npm run lint
npm test # Vitest with coverage
npm run build # CDS types + TypeScript
# Real SDK, no chain access
npm run smoke:sdk
npm run integration:providers # also: wallet-keys, wallet-facade, contract-registry,
# connector-routes, attestation-vault, derive-wallet-info
# Live e2e on preprod (funded wallet)
npm run deploy:e2e # more lanes: docs/operations.md
License
Links
History
Fixes 1
Notes
Fixes
@odatano/nightgate-tx0.14.2: the README example ofconnect()passesdocumentandproofFieldsas in 0.36.0. The builder code is unchanged;connect()forwards action parameters as given.
Changed 11, Other
Notes
Proof inputs as typed OData parameters. BREAKING for callers of the proof actions: the
*Jsonstring parameters are gone, the same values go as arrays and objects. No schema change, no circuit change.Changes
issueFieldPredicateAttestation,issueFieldEqualityAttestation,issueFieldMembershipAttestation:siblings: array of Hex64anddirs: array of BooleanreplacesiblingsJson/dirsJson. Membership takesallowedValues: array of StringorsetRoot+setSiblings+setDirs.issueFieldPredicateAttestationBatch:claims: array of BatchClaimreplacesclaimsJson; the responseclaimsis an array ofIssuedClaim.issueDocumentIntegrityAttestation,issueDocumentDiffAttestation:schema: array of SchemaSlot,openingAandopeningB: DocumentOpeningreplace the three JSON strings.prepareDocumentProof(document: Map, proofFields: array of ProofField)replacesdocumentJson/proofFieldsJson;fields,emptyFields,schema,leavesandopeningcome back as arrays and an object.prepareMembershipSet(allowedValues: array of String)replacesallowedValuesJson; the path comes back assetSiblings/setDirs, empty withoutvalue/valueDigest.- Types
ProofField,PreparedField,SchemaSlot,SlotOpening,DocumentOpening,BatchClaim,IssuedClaimin$metadata. CAP answers 400 before the handler runs for a non-array, a wrong element type, an enum value outside the list, a number outside its range, a malformed hash inside a structured type, a string where an object is expected and an unknown parameter name. deployContract,submitContractCall,submitContractCallBatch:initialPrivateState: Mapreplaces the JSON string.postSwapOffer:tags: array of Stringreplaces the JSON string.predicateonissueFieldPredicateAttestationandverifyPredicateStateandkindonpurgeExpiredare enums in$metadata; the values are unchanged.submitContractCallBatch.calls,args,anchorDocument.metadataandattestAgentOutput.envelopeJsonstay JSON text: circuit arguments are untyped, metadata and envelope are the hashed bytes.@odatano/nightgate-mcp0.16.0 sends the new shapes.
Migration
before after siblingsJson: '["…"]',dirsJson: '[true,…]'siblings: ["…"],dirs: [true, …]setSiblingsJson,setDirsJsonsetSiblings,setDirsallowedValuesJson: '["EEA"]'allowedValues: ["EEA"]claimsJson: '[{…}]'claims: [{…}]schemaJson,openingAJson,openingBJsonschema,openingA,openingBdocumentJson: '{…}',proofFieldsJson: '[{…}]'document: {…},proofFields: [{…}]initialPrivateState: '{…}'initialPrivateState: {…}tags: '["…"]'tags: ["…"]responses fields,emptyFields,schema,leaves,opening,claims,setSiblingsJson,setDirsJsonas JSON textarrays and objects; setSiblings,setDirsChanged 1
Notes
Changed
Dockerfilebuilds both stages fromnode:22.23.4-slim(Node security release of 2026-10-21: Brotli decoder out-of-bounds write, TLS hostname checks, keep-alive response desync, dependency updates). The base image is pinned to the exact Node version instead of the floatingnode:22-slim. No code change.
Fixes 1
Notes
Fixes
@odatano/nightgate-tx0.14.1: the transaction builder imports@midnight-ntwrk/compact-runtimeinstead of requiring it. Under a loader that servesrequire()of an ESM package separately (tsx), theapibuilder's contract state came from a second runtime instance and every circuit call failed with'contractState' parameter ... has unexpected type. Plain Node was not affected.
Deprecated 1, Changed 5
Notes
Services under
/odata/v4/midnight…; the/api/v1/…paths keep answering until 1.0. No schema change, no circuit change.Changes
- Service paths:
NightgateService/odata/v4/midnight,NightgateIndexerService/odata/v4/midnight-indexer,NightgateVerifyService/odata/v4/midnight-verify,NightgateAnalyticsService/odata/v4/midnight-analytics,NightgateAdminService/odata/v4/midnight-admin. Names, model and handlers unchanged;/contract-manifestand/zk-config/*stay where they are. - The agent-token lane and the public verify lane (CORS
*) sit on the new paths. - Container healthcheck and watchdog read
/odata/v4/midnight-indexer/getReadiness(). getMetrics():legacy_path_requests_total{path}, requests on each old path since process start.@odatano/nightgate-tx0.14.0:connect()defaultsservicePathto/odata/v4/midnightand the newindexerServicePath(getHealth()) to/odata/v4/midnight-indexer. Against a server below 0.35.0 passservicePath: '/api/v1/nightgate', indexerServicePath: '/api/v1/indexer'.
Deprecated
/api/v1/nightgate,/api/v1/indexer,/api/v1/verify,/api/v1/analytics,/api/v1/admin: rewritten to the new paths before the services (srv/utils/legacy-service-paths.ts), logged once per path, counted ingetMetrics(). They go with 1.0.
- Service paths:
Changed 8, Fixes 2
Notes
One throughput policy: rate classes, grant budgets per hour, one 429 answer. Schema delta: three nullable columns on
AgentGrants. No circuit change.Changes
- Rate classes replace the per-action limiters. Each class has a fixed window
and one count,
NIGHTGATE_<CLASS>_RATE_LIMITorrateLimits.<class>inNIGHTGATE_SPONSOR_POLICY_FILE, which applies without a restart:prove60/hour per session (contract calls and batches, token operations, theissue*attestations counting their claims, anchoring, document registry, disclosure,buildSponsorable),deploy5/hour per session,sponsor120/hour (the three sponsor actions andmintFactoryToken),compute120/hour (prepareDocumentProof,prepareMembershipSet,claimDisclosure,postSwapOffer,retireSwapOffer,reindexDisclosuresper contract),read60/minute (offer board,getBoardStatus,getRuntimeInfo, wallet diagnostics),wallet10/minute (connectWallet,sendNight, dust registration),signing10/hour,admin10/hour,public-verify60/minute per client address.NIGHTGATE_GRANT_ADMIN_RATE_LIMITisNIGHTGATE_ADMIN_RATE_LIMIT. The old numbers: deploy 5/h, call 30/min, anchor 10/h, predicate 10/h, disclosure 30/h, registrar 30/h, reindex 60/h, sponsor 120/h, swap offer 60/h, swap list 600/h, holder claim 60/h, build 30/h, prepare 120/h, board status 60/min, runtime info 30/min, connect 10/min, send 10/min, dust registration 10/h, diagnostics 60/min. AgentGrants.maxJobsPerHournext tomaxJobsPerDay: a budget per UTC hour that survives a restart, oncreateAgentGrant,createAgentGrantsandupdateAgentGrant; a unit goes back when the request fails with a 4xx, a rate class that refuses after the take included.NIGHTGATE_GRANT_DEFAULT_JOBS_PER_HOURand_PER_DAYfill a budget the creation leaves out.sponsorFinalizedTransaction,sponsorUnboundTransactionandsponsorSwapdeserialize the caller's bytes before a job exists: input that is not a proven transaction answers 400 at once instead of failing in a proving slot.- Every 429 names its limit and carries
Retry-After:Rate limited (prove: 60 per hour). Retry after 58s. Answers under a limit carryRateLimit-PolicyandRateLimit(IETF httpapi draft). getGrantUsagereturnslimits: the hour and day budgets as{ max, used, resetsAt }and the rate classes a grant can reach as{ name, windowSeconds, max, remaining }. A token may leavegrantIdempty.getMetrics():rate_limited_total{limit}.- Wallet worker pool.
NIGHTGATE_WALLET_WORKERS(default 1) runs user wallets on several threads, a wallet always in the same one;NIGHTGATE_SPONSOR_WORKERS=dedicatedgives every pool sponsor a thread of its own, warms the sponsors side by side at boot, and lets sponsored jobs of different sponsors run in parallel (one thread carried every sponsor before: ~20 sponsored jobs a minute on the hosted box). Saves are ordered per wallet, a crash or rotation of one worker touches only its wallets. A bound call whose sponsor lives in another worker answers 412; keepsharedfor bound sponsoring on pool wallets.getMetrics()and the status report the pool and every worker. @odatano/nightgate-tx0.13.0:NightgateApiError.retryAfterSecondsfromRetry-After.
Fixes
sponsorUnboundTransactionandsponsorSwapkeep the dust backing reserved after the spend went out until the sponsor wallet no longer lists the spent note (NIGHTGATE_NOTE_SPENT_WAIT_MS, default 30 s). The reservation ended at inclusion before, and the next job on that backing built a second spend of the same note, which the node rejected (1010/196) and the job rebuilt.- A submit that died on the client's own closing socket counts as a resend: a copy the node refuses is looked up on the indexer and reported as landed, instead of being rebuilt as a dust race. The resend, refusal and dust-race log lines carry the transaction identifier.
- Rate classes replace the per-action limiters. Each class has a fixed window
and one count,
Fixes 1
Notes
Fixes
@odatano/nightgate-tx0.12.2: a builder onapideserialized the contract state with the ledger class; circuits run on compact-runtime's class of the same name and refused it ('contractState' parameter has unexpected type).apiPublicDataProvidertakesruntime(compact-runtime) for the contract state andledgerfor the zswap state and the ledger parameters.
Fixes 1
Notes
Fixes
- Binary columns updated on PostgreSQL (
ContractStates,ContractActionsstate and zswap state,Blocks.ledgerParameters) were stored as base64 text and read back encoded twice;stateAtserved them that way and a builder on the API failed withexpected header tag 'midnight:contract-state[v6]:'. Writes hand the driver bytes now. Rows written before: one SQL run, seedocs/operations.md.
- Binary columns updated on PostgreSQL (
Changed 1
Notes
Changes
@odatano/nightgate-tx0.12.1:maxLagBlocksdefaults to 60. A server's contract states trail the chain by its indexer pass, some ten blocks in normal operation, which the old default of 10 refused.
Changed 5
Notes
Changes
ContractStates/stateAtreturnsblockHash, theledgerParameters(base64) in effect at the state's block,coveredHeight(the database holds every contract state up to this block) andchainHeight. The parameters come fromBlockswhen the indexer supplement stored them, else from the indexer once; a used-up indexer budget is a 429,nullwhen the indexer cannot answer.Transactions/byIdentifier(identifier): the transaction a ledger identifier belongs to, ashash,ledgerTxHash,blockHeight,blockHash,statusandfailedSegments. 404 while the indexer has not seen it. Allowed under every agent token.- Schema: table
TransactionIdentifiers(one row per identifier and transaction), written by the decode pass, removed by a reorg rollback, filled for existing rows in the background at the first start (SyncState.identifiersBackfilledAtmarks the finished run). @odatano/nightgate-tx0.12.0:createTxBuilder({ api })reads contract state and the deploy through a NIGHTGATE server instead of an indexer and a node (walletSync: false; withapithe builder dials no node);probeLandedandwaitLandedtake{ api }(a refused request is thrown, only an outage reads as unknown);maxLagBlocks(default 10) refuses a state from a server further behind the chain;connect()gainscontractStateAt,transactionByIdentifierandcontractDeploy.- A sponsor action that cannot read the handed-over transaction names the base64 characters received, the bytes decoded and the full reason of each attempt.
Breaking 2, Changed 2
Notes
Breaking
sponsorFinalizedTransaction,sponsorUnboundTransactionandsponsorSwaptake nosponsorSessionId. The platform pool (NIGHTGATE_FEE_SPONSOR_SESSION) pays; a grant pinned to a wallet session (createAgentGrantsponsorSessionId) pays from that session. A request that still carries the parameter is a 400.- Without a pool and without a pinned grant the three actions answer 503.
Changes
getJobStatusunder any agent grant may poll pool jobs, under the reserved pool session id the sponsor actions return.@odatano/nightgate-tx0.11.0:sponsorFinalized,sponsorUnboundandsponsorSwapsend no sponsor session; the poll uses the session the server returns.
Changed 3
Notes
Changes
- Dust snapshot check budget (
NIGHTGATE_DUST_COLLAPSE_BUDGET_MS) counts from the moment the helper thread begins a check and defaults to 120 s. A check past it ends only that one: checks waiting behind it move to the restarted helper instead of failing with it. - Save ticks of all wallets run one after another, so the wallet thread collapses one dust state at a time and the helper checks one at a time. A timer that fires while its wallet's previous tick still runs is skipped.
- Worker thread RPC: the worker announces the start of a call on its port.
- Dust snapshot check budget (
Changed 5
Notes
Changes
- Dust snapshot collapse: the restore check of the collapsed state runs in a
helper thread of the wallet worker; the wallet thread only collapses and
serializes. A check past
NIGHTGATE_DUST_COLLAPSE_BUDGET_MS(30 s) saves the full snapshot once and restarts the helper. - A save tick skips the dust part while the dust state is unchanged since the last confirmed save (applied ledger event, coin and pending coin counts).
getMetrics()gainsdust_collapse_ms,dust_collapse_p99_msanddust_collapse_runs_total{outcome}per wallet; a collapse over 5 s is logged at INFO.scripts/profile-workers.mjs: CPU profile of the main thread and every worker thread through the inspector.- One worker-thread RPC module (
srv/midnight/worker-rpc/) serves the decode worker and the new helper.
- Dust snapshot collapse: the restore check of the collapsed state runs in a
helper thread of the wallet worker; the wallet thread only collapses and
serializes. A check past
Changed 6
Notes
Changes
getReadiness()reports aphase(starting,warming,catching-up,ready,degraded,failed) and staysreadywhile the sponsor pool warms; the database read gives up afterNIGHTGATE_READINESS_DB_TIMEOUT_MS(2 s) instead of queueing behind a saturated pool.- Contract state for
verifyAttestationState,verifyPredicateState,verifyDocument,verifyPredicateAttestation,claimDisclosureand the disclosure reindex, and the crawler's ledger payload decoding, run in a second worker thread (NIGHTGATE_DECODE_WORKER_RPC_TIMEOUT_MS). The main thread no longer runs wasm. - Snapshot decryption derives keys off the event loop; a carried snapshot already under the account key is kept as is (was re-derived and decrypted on every save).
- Event-loop lag is logged above
NIGHTGATE_EVENT_LOOP_LAG_WARN_MS(500 ms) with the pool wait depth;getMetrics()gainsevent_loop_lag_p99_ms,event_loop_lag_max_msand thedecode_worker_*gauges. docker/watchdog.shskips restarts forNIGHTGATE_WATCHDOG_START_GRACE(900 s) after a container start and logs each skip.scripts/presync-wallets.mjssyncs sponsor wallets on any machine;scripts/import-wallet-state.mjsloads the states into a stopped server.
Changed 3, Other 6
Notes
Changes
- 32-byte hex parameters are typed
Hex64: CAP rejects a malformed value with400 ASSERT_FORMAT("must be 64 hex characters") before the handler runs. AlsoanchorDocument.sha256andverifyDocument.providedSha256. - Results carry every declared field,
nullwhere it does not apply (was left out):prepareMembershipSetwithout a member,claimDisclosure,getSponsorPolicy(unset lists are[]). estimateSendNightFeerejects a missingamountlikesendNight.
Schema
Transactions.proofHashremoved. PostgreSQL needs a step before and after the first boot, seedocs/operations.md(Schema upgrades).NOT NULLwith their defaults:BackgroundJobs.status,attempt,maxAttempts,PendingSubmissions.status.nightgate-schema-deltafills NULL rows with the default.- Enums on unchanged columns:
SwapOffers.status,BackgroundJobs.chainStatus(incl.dropped),BackgroundJobs.commandEncoding.
Development
- cds-typer in its default output with
propertiesOptional: falseandIEEE754Compatible: true; handlers register with the generated action constants. npm run lint:no-explicit-anyis an error outside a shrinking per-file baseline;scripts/check-conventions.mjsguards typed models and comments.@odatano/nightgate-tx0.10.3: doc comments, shipssrv/utils/errors.js.
- 32-byte hex parameters are typed
Fixes 1
Notes
Fixes
listSwapOffersandgetSwapOfferreturntagsas an array of strings (many String), no longer as JSON text.postSwapOfferstill takestagsas a JSON array in text.
Features 2
Notes
Features
@odatano/nightgate-tx0.10.2, the token factory on the caller's side:tokenFactoryIssuerSecret({ seedHex })derives the issuer secret of a seed by the rule the server applies to a session on the same seed;prepareMint,prepareBurn,tokenTypeOf,tokenName,TOKEN_FACTORY_CIRCUITSre-exported from@odatano/contract-kit. A mint built withcreateTxBuilder({ package: '@odatano/contract-token-factory' })goes throughsponsorUnboundTransactionlike any sponsorable call.- The issuer rule lives in
@odatano/contract-kit0.1.1 (deriveTokenFactoryIssuerSecret); the server derives a session's issuer secret through it. Issuer keys unchanged.
Features 5
Notes
Features
- Offer board reads:
getSwapOffer(offerId)(one offer, open or closed);listSwapOfferstakesstatus(opendefault,filled|retired|expired|all, closed ones ordered by last change),since(offers changed after an instant: the board's change feed; an offer the clock ran out counts as changed atexpiresAt) andmine(the caller's own posts, a token's grant); every row carriesstatus,filledTxHash,closedAt,changedAt. Admin projectionSwapOfferswith the posters.getSwapOfferis open to every token. getBoardStatus()on the indexer service, anonymous like the probes: open offers, offers filled and sponsored swaps succeeded today, sponsors configured and ready (from the worker's last pushed reading, no worker call). Computed at most every 10 s; 60/min per client.createAgentGrants(count, sessionId, allowedActions, labels?, ...): 1 to 50 grants of one shape on one session in one call, tokens returned once, labels given or<agentLabel | 'agent'>-1..n; same checks and rate limit ascreateAgentGrant.mintFactoryToken(contractAddress, name, amount, recipientCoinPublicKey, sessionId, idempotencyKey?, sponsorSessionId?)on thetoken-factorylineage: the session is the issuer (issuer secret derived from its seed, witness held by the worker), the result namesissuerKey,domainandtokenTypebefore the job runs, a landed mint is recorded inLearnedTokenTypesand on the grant, also when the indexer proves it after a lost broadcast.LearnedTokenTypesholds at most 256 types. Grantable (mintFactoryToken, circuitmint). CodeTOKEN_FACTORY_UNAVAILABLE(404).@odatano/nightgate-tx0.10.1:createTxBuilder({ package, from? })takes class, name, private-state id and keys directory from an installed lineage package;ensureZkAssets({ package })fetches its missing prover keys from the package's release assets, verified againstkeys/manifest.json(a givenzkConfigBaseUrlstill wins);resolveBuilderPackageexposes what it reads.zkAssets.source: 'package'.
No schema change, no circuit change.
- Offer board reads:
Features 6, Changed 2
Notes
Features
- Contract lineages as npm packages, repository
ODATANO/NIGHTGATE-CONTRACTS:@odatano/contract-attestation-vault,-attestation-vault-32,-holder-registry,-shielded-token,-token-factory,-counter, each withcontract.json(role, digest, compiler and runtime versions, prover-key source) and the Compact source; companion code (hex, hashing, content roots, set roots, claim keys, witnesses, call helpers, artifact digest, key manifest, token-factory names and calls) in@odatano/contract-kit.token-factoryis a sixth registered lineage (shielded tokens on demand, named by their issuer, with a supply ledger and burn; served under/zk-config/token-factory).cds.requires.nightgate.contractsentries take{ "package": "<name>" }; the path form stays for foreign artifacts. Prover keys are fetched from the package's release assets (contract.json#zkAssetUrl, flat layout);nightgate-fetch-keys <name|package>prefetches them, the image does so at build.getRuntimeInfo.contracts[]carriespackageandversion. The generation check runs before any key fetch. - Offer board:
postSwapOffer(offer, expiresAt?, tags?),listSwapOffers(givesType?, wantsType?, tag?, limit?),retireSwapOffer(offerId). A posted maker half is checked like a sponsored half, its terms read from the half; it closes when one of its input nullifiers lands in a sponsored swap, on expiry, or by its poster.sponsorSwaptakesofferIdinstead ofmakerHalfB64. TableSwapOffers; codesSWAP_OFFER_INVALID(400),SWAP_OFFER_NOT_OPEN(409); rate limits 60/hour (post, retire) and 600/hour (list). Grantable, also on a pool grant; listing is open to every token. - Token types learned platform-wide:
NIGHTGATE_SPONSOR_SHARE_MINTED_TOKEN_TYPES/ policy fileshareMintedTokenTypes(needsallowContractMints) lists every type a landed sponsored mint created for every grant; a grant may narrow to a subset. TableLearnedTokenTypes, read-only admin projection. - Disclosure to token holders: contract
holder-registry(registerHolder(coin, claim_key)passes a coin of the type through the contract and recordsholderEntry(type, claim_key);unregisterHolder), actionsgrantDisclosureToHolders(payloadHash, tokenType, registryAddress, content?, contentType?, expiresAt?),revokeHolderDisclosure(holderGrantId),claimDisclosure(payloadHash, tokenType, claimSecret). Content is stored encrypted and must hash to the payload; a claim reads the registry live. TableHolderDisclosureGrants. txbuilder:holderClaimKey,HOLDER_REGISTRY_CIRCUITS. - Worker:
describeSwapHalfRPC; a sponsored swap's result and submit intent carry the halves' input nullifiers. @odatano/nightgate-tx0.10.0: depends on@odatano/contract-kitand the two vault packages instead of shipping copies;./attestation-vault[-32]re-export them.
Changes
contracts/left this package;./browser/attestation-vault[-32]re-export the lineage packages for one release line.hashing,set-root,predicate-stateclaim keys,document-proofbuilders,contract-witnesses,artifact-digestand the browser helpers are re-exports of the kit. Scriptscheck:vault-parity,keys:manifest,test:contractand the circuit integration tests moved to the contracts repository.NIGHTGATE_CONTRACTS_DIRdefaults to the working directory'scontracts/only.- Generation digests of
attestation-vault,attestation-vault-32andholder-registryare unchanged;counterandshielded-tokenwere recompiled from their sources and carry new digests.
Schema delta: three tables, no column change;
nightgate-schema-deltaon SQLite,cds deployon PostgreSQL. No circuit of the vault lineages changed.- Contract lineages as npm packages, repository
Fixes 1
Notes
Fixes
- The wallet worker's indexer tip read (the sync gate's freshness check) falls back to
the indexer's
blockssubscription when the HTTP query fails, then to the last successful read withinNIGHTGATE_INDEXER_TIP_GRACE_MS(default 3 min,0= off); the reused timestamp still ages againstNIGHTGATE_SYNC_FRESHNESS_MS. A refused or timed-out HTTP query alone no longer takes a synced sponsor out of the pool.getWalletSyncProgressreportsindexerErrorwithindexerFresh: truefor a reuse; the worker logs each change of the read path once.
- The wallet worker's indexer tip read (the sync gate's freshness check) falls back to
the indexer's
Features 1, Fixes 2
Notes
Features
- Admin
redecodeFromHeight(height): lowers the ledger payload decode cursor so the decode pass replays the indexed blocks fromheightand rewritestxType,isShielded,hasProof, the counts and circuit names from the payloads. Returns{ fromHeight, previousDecodedHeight, blocks, changed }; the cursor is never raised.
Fixes
- A replayed regular transaction whose payload does not decode ends with
isShieldedandhasProofnull(was: kept a storedfalse). - Job code
1010/103(shielded offer invalid: a coin already spent, a stale zswap proof) carries its meaning in the message; a swap offer filled twice ends there, nothing is rebuilt.
- Admin
Features 5, Fixes 4, Dependencies 1, Other 1
Notes
Features
sponsorSwap(makerHalfB64, takerHalfB64, sponsorSessionId, idempotencyKey?): the sponsor pays the dust of a shielded swap handed over as two proven halves, bound or unbound, each as offer file (bech32m,swapoffer1...) or base64. Each half gives one token type and wants another, both inallowedTokenTypes; the halves mirror each other. Off by default:NIGHTGATE_SPONSOR_ALLOW_SWAPS/ policy fileallowSwaps, andsponsorSwapin a grant'sallowedActions. Job result{ txHash, swap: { gives, wants } }.NIGHTGATE_SPONSOR_SWAP_MAX_INPUTS(4).NIGHTGATE_SPONSOR_ALLOW_CONTRACT_MINTS/ policy fileallowContractMints(default off): the sponsor pays the offer of a token a call of the same transaction mints, without anallowedTokenTypesentry. The contract has to be sponsorable, the offer creates at most the declared amount.- A landed mint under a grant records the type (
AgentGrants.mintedTokenTypes); it counts as listed for that grant while contract mints are sponsored. - Admin
getSponsorPolicy(grantId?): platform policy, its source and load time, the grant's lists, the effective policy or the reason it is empty. @odatano/nightgate-tx0.8.0 (txbuilder):createSwapWallet: shielded-only wallet,buildHalf,takeOffer,revert,serializeState/walletState; coin selection withinmaxInputs(4),coins(),spendable(tokenType).encodeOffer,decodeOffer,readSwapTerms,sameSwapTerms.createTxBuilder:walletSync: 'shielded',walletState,waitForSync(),serializeWalletState(),shieldedKeys.buildSponsorable/buildDeploySponsorable:recipientsfor coins created for another wallet.deriveIdentityreturnsaddresses.shieldedandshieldedKeys;deriveRoleSeeds.- Client
ng.sponsorSwap(...).
Fixes
createAgentGrant/updateAgentGrantanswer 400 for contract or circuit lists that share nothing with the platform's and for a token type the platform does not list (were accepted, the grant then failed at its next sponsored call).- Grant token types that share nothing with the platform's sponsor no offer; the
grant's calls without an offer keep working (was
403 SPONSOR_POLICY_EMPTYon every sponsored call). - A WARN names the sponsor env settings a policy file replaces.
Transactions.txTypeof a transaction without contract action or unshielded movement isshielded_transferordust_registrationonce decoded (crawler.decodePayloads; wascontract_call).
Schema
AgentGrants.mintedTokenTypes; additive, probed at startup (nightgate-schema-delta).
Dependencies
@scure/base^2.2.0 (bech32m), already in the tree through the wallet SDK.
Fixes 3, Other 1
Notes
Fixes
Transactions.isShieldedandhasProofcome from the decoded ledger payload (crawler.decodePayloads): zswap inputs, outputs or transients; a contract call or zswap proof. Both arenulluntil a row is decoded (were alwaysfalse).Transactions.proofHashis no longer set (was the extrinsic hash).palletMapisShieldedapplies only until the payload is decoded.
Schema
Transactions.isShielded/hasProoflose theirfalsedefault; no migration.
Features 1, Fixes 3
Notes
Features
- Admin
revokeRole(userId, role, scope?): ends active disclosure-role grants (validUntil= now, rows kept), returns the number ended; caller needsauthority.
Fixes
grantRolerejects avalidUntilthat is not a timestamp or not in the future (400).verifyDocumentreturnsoriginalSha256only when the provided hash matches, else''.- A worker RPC waits at most
NIGHTGATE_SUBMIT_INTENT_ACK_TIMEOUT_MSfor pending submit-intent hooks before it answers the caller.
- Admin
Breaking 1, Features 6, Fixes 5, Other 1
Notes
Breaking
error.codeof a reject that names only a status is a string:INVALID_ARGUMENT,NOT_FOUND, ... instead of"400"; in-process callers (srv.send) read the number fromstatus. HTTP statuses are unchanged.
Features
- One error registry (
docs/reference.md#http-error-codes,npm run errors:table); error classes carry their code, also across the wallet-worker boundary. A 5xx keeps its code in production; its body is reduced to code and message. getJobStatus.chainSegmentsfor confirmed batches:[{ segment, calls, applied }].- Admin
reconcileNightBalances(address?, after?, limit?): drift report, no writes. - One process per database runs jobs, restart recovery and crawler (
InstanceLeases,NIGHTGATE_INSTANCE_LEASE_TTL_MS, 90 s). A process whose lease was taken over dispatches no job, refuses write actions and broadcasts, and reports offline. - A timed-out worker RPC is cancelled at the worker's next wait point (sync, backing, before proving); never after the submit intent.
@odatano/nightgate-tx0.7.0:isRetryable(err),RETRYABLE_ERROR_CODES.
Fixes
- A wallet not at the tip before the build fails retryable (
NetworkOrTimeout,wallet-not-synced) and a sponsored job fails over (wasSubmitAmbiguous). - A job refused by a revoked grant or the sponsor policy fails with
AGENT_GRANT_REVOKED/SPONSOR_POLICY_UNAVAILABLE/SPONSOR_POLICY_EMPTY(wasError). - Missing prover keys answer 503
PROVER_KEYS_UNAVAILABLE(was a 500). BlockStatisticscounts per returned row; default page 100, max 1000.- The crawler's error counter increments in the statement; spent UTXOs are looked up once per block.
Schema
BackgroundJobs.chainSegments, new entityInstanceLeases; additive, probed at startup.
Fixes 16
Notes
Fixes
- A sponsor held back by the indexer freshness check says why:
indexer unreachable (HTTP 403)(ortimeout) when the tip read failed,indexer's newest block is Ns oldwhen it lags. A failed read used to be reported asindexer not freshlike a lagging indexer; an answer from the indexer's edge that is not JSON is recognised as a refusal. getWalletSyncProgressgainsindexerTipAgeSecondsandindexerError; the worker's idle-sync log line carriesindexerError.- Disconnecting a session no longer evicts the wallet's facade while another user's session that holds the signing key uses it; another user's viewing-only session still never keeps the keys in memory.
- A worker RPC whose message cannot be posted settles at once instead of holding its entry until the timeout.
- Node notifications buffered for subscription ids that are never registered are capped (16 per id, 32 ids) and dropped on unsubscribe.
- The SCALE parser skips 65 bytes for an ECDSA extrinsic signature (was 64, which misread the rest of such an extrinsic).
- A fork search deeper than 100 blocks throws instead of returning an unverified height; nothing is rolled back and the next head retries.
CapDbPrivateStateProvider.clear()removes the current contract's private states only, likeget/set/removeand the SDK's own provider (it removed every contract's states of the account).grantDisclosurereplayed under its idempotency key restores thependingLevelit found; the replay starts no job, so nothing cleared it.- An
idempotencyKeyover 128 characters is refused with 400IDEMPOTENCY_KEY_INVALID(was a database error, 500 on PostgreSQL). attestAgentOutputrecords itsanchorDocumentjob under the caller's agent grant (revocation and per-grant usage cover it).- A worker exit counts as a rotation only for the worker that announced it; a stopped worker's announcement no longer hides the next worker's crash.
- The worker's deploy-query cache is bounded (256 entries).
NIGHTGATE_JOB_LEASE_TTL_MSis at least 60 s (twice the heartbeat); a shorter lease reclaimed running jobs.profileWorkerwrites only insidenightgate-profiles/in the OS temp dir;dirnames a folder there, a path outside it is a 400.- Reconciliation lookups that fail log at debug level instead of silently.
- A sponsor held back by the indexer freshness check says why:
Fixes 1
Notes
Fixes
- Tests: the predicate handler tests compare each proof argument exactly; a time-derived argument containing the field value's digits failed them.
Breaking 1, Features 3, Fixes 1, Other 1
Notes
Breaking
- The indexer supplement no longer stores the full contract state on every
ContractActionsrow. Each action getsstateHash/zswapStateHash(sha256) andstateSize/zswapStateSize; the full state stays only undercrawler.contractStateHistorywatched(addresses incrawler.contractStateWatch) orall(NIGHTGATE_CRAWLER_CONTRACT_STATE_HISTORY,_WATCH). Defaultnone.
Features
ContractStates: the newest state per contract, written by the supplement (a higher block wins, the same block replaces); a rollback drops the rows at or above the fork height.ContractStates/stateAt(address, height): the state after the contract's newest action at or belowheight, fromContractStates, a kept per-action state, or the indexer (supplement URL, thenNIGHTGATE_INDEXER_HTTP_URL), withsourceandverified(indexer bytes against the stored hash). Above the crawler's indexed tip, or when the newest action at that height has no hash yet,verifiedis null. Indexer reads are capped at 2 per second across callers (429 +Retry-After).nightgate-compact-contract-state(scripts/compact-contract-state.mjs,--dry-run,--batch): moves stored per-action states to the new layout, honours the history setting, idempotent.
Fixes
- The payload decoder marks
SYSTEMtransactionsabsentinstead of trying to deserialize them as a ledgerTransaction, which failed on every one and logged a warning each.
Schema
- New columns on
ContractActions, new entityContractStates; additive (cds deployon PostgreSQL,nightgate-schema-deltaon SQLite). The startup preflight probes both.
- The indexer supplement no longer stores the full contract state on every
Breaking 5, Fixes 17
Notes
Breaking
- A v1/v2 ciphertext in a column bound to a purpose and a row (wallet session
keys, account keys, job commands) is refused (
UnboundEnvelopeError); runnightgate-rewrap-keysbefore the upgrade, or setNIGHTGATE_ACCEPT_UNBOUND_ENVELOPES=trueuntil it ran. The tool itself still reads them. auth.kind: dummyrefuses to start in production unlessNIGHTGATE_ALLOW_UNAUTHENTICATED=true.- Image: the basic-auth user carries no role unless
NIGHTGATE_HTTP_ROLESnames one (wasadmin); setNIGHTGATE_HTTP_ROLES=adminfor the admin service. - Compose publishes port 4004 on
127.0.0.1only (NIGHTGATE_BIND_ADDRESS=0.0.0.0restores the old binding). AttestationServicetiers (Public,Disclosed,Authority) require an authenticated user, also in a host projection without the tier handlers.
Fixes and hardening
- Concurrent resolutions of one account key each get their own buffer
(
resolveAccountDek); a caller that zeroes its copy after use no longer zeroes the key of the others. - Contract deploys, calls and batches read private states still under a legacy
password: the worker-routed provider gets
privateStoragePasswordFallbacks. invalidateSession/invalidateAllSessionsdeactivate the session first and evict the facade under the facade build lock, so a build in flight cannot put the keys back.- The crawler never stores a block with extrinsics whose
System.Eventsare empty or undecodable: empty is a transient error (retried), undecodable a permanent one (the block latches). Such a block used to be stored without its UTXO rows and balance deltas. - A crawl pass that fails on a transient error records
lastErrorand runs again after 30 s; finalized head and header reads go through the retry. A permanent failure setssyncStatustoerror, andresumeCrawlerrestarts a crawler that stopped that way (it used to answer "already running"). - Ledger error
171(OutOfDustValidityWindow, a lagging node) is a dust race: rebuilt and resubmitted like170and196. - A workflow step that is on chain but whose record fails after lock-contention
retries parks the workflow as
reconciliation_requiredinstead of failing it; the re-run reuses the landed step. - An idempotency key reused with a different payload answers 409
IDEMPOTENCY_KEY_CONFLICT(was a masked 500). attestAgentOutputretried under the sameidempotencyKeywithoutproducedAtreuses the first call'sproducedAt(recorded in the anchor job's request), so the retry yields the same envelope and job instead of a 409.- Heavy job kinds share ONE concurrency cap (
jobs.concurrency.heavy, 4) instead of 4 per kind; workflow parents have their own cap of the same size, so a parent never holds a slot its children need. - A wallet build that outlives the worker RPC timeout is joined by the next
initinstead of started twice; an evict during the build zeroes the finished keys instead of registering them. - The node connection pings every 30 s; a socket without an answer for one interval is closed and reconnected.
- Rollback and
reindexFromHeightquery and delete in chunks of 5000 ids (PostgreSQL caps a statement at 65535 parameters). - Indexes:
UnshieldedUtxos.createdAtTransaction_ID;Blocks.heightbecomes unique (ng_blocks_height_uniquereplacesng_blocks_height; with duplicate heights the old index stays and startup warns). - SCALE compact decoding reads the four-byte mode unsigned.
- The node URL in the unreachable-node warning is logged without credentials.
- Version in
package.json, lock and the compose default tag.
- A v1/v2 ciphertext in a column bound to a purpose and a row (wallet session
keys, account keys, job commands) is refused (
Other 4
Notes
NIGHTGATE_DUST_SNAPSHOT_COLLAPSE=truesaves the dust snapshot with every generation-tree leaf collapsed that does not back one of the wallet's own dust UTXOs (night_indices). The ledger re-expands foreign leaves on each dtime update and never collapses them again, so the snapshot and its restore (DustLocalState.deserialize, one synchronous wasm call) grew with the chain: 6 MB and 5 to 7 minutes on preprod, 7 KB and 4 s with the flag. Roots, balance and spends are unchanged. Default off.- The collapsed state is restored once before it is saved and must give the
same roots, balance and UTXO count; otherwise, and on any error in that path,
the save keeps the SDK snapshot and logs the reason once
(
srv/midnight/worker/dust-collapse.ts). - Snapshot state and
offsetcome from one emitted wallet state, so a restore replays from the offset that matches the saved state. - Version in
package.json, lock and the compose default tag.
Other 4
Notes
getSponsorPoolStatuskeeps showing a sponsor while the wallet worker does not answer (e.g. while it restores another wallet's dust snapshot, a single synchronous call of several minutes): the row carries the last dust figures the worker pushed,stale: trueandasOf,usable: false,unshieldedNight: null,lastErrorwith the reason. It used to readdustNotes: 0for every sponsor. A live read answersstale: falsewith the read time asasOf.- The worker attaches
dustfigures (balance, free and pending notes, restore count, NIGHT UTXO counts,at) to the sync snapshot it pushes when a facade reaches the tip and on every progress-watch tick (dustFiguresOf); a push without them keeps the previous ones. - Two additive fields on the
getSponsorPoolStatusreturn type; no schema change, no migration, no config change. - Version in
package.json, lock and the compose default tag.
Other 2
Notes
- A parked job whose kind has a reconciliation finalizer and whose command is
stored encrypted (
aes-gcm-v1) resolves again: the finalizer decrypted the command without its job binding (jobCommandBinding), the decrypt threw and the job stayedreconciliation_requiredalthough the indexer had its outcome. AffectssponsorUnboundTransaction,sponsorFinalizedTransaction,submitContractCallBatch,anchorDocument,grantDisclosure,revokeDisclosure,registerPassport,retractandfieldPredicateBatchProofsince 0.23.4. Parked jobs resolve on the next confirm pass after the upgrade. - Version in
package.json, lock and the compose default tag.
- A parked job whose kind has a reconciliation finalizer and whose command is
stored encrypted (
Other 5
Notes
NIGHTGATE_INDEXES(srv/utils/db-indexes.ts) gains five indexes the indexer supplement updates and deletes through:TransactionFees.transaction_ID,TransactionSegments.transactionResult_ID,ContractBalances.contractAction_ID,ZswapLedgerEvents.transaction_ID,DustLedgerEvents.transaction_ID.- Four for newest-first reads:
Blocks.createdAt,Transactions.createdAt,ContractActions.createdAt,Transactions(txType, createdAt); DESC, on PostgreSQLDESC NULLS LASTto match the ORDER BY of the nullable column (IndexSpec.postgres,indexStatement(spec, kind)). Serves$orderby=createdAt desc,byTypeandhistory. - Created at start with IF NOT EXISTS. On a large live PostgreSQL create all
nine once with
CREATE INDEX CONCURRENTLYfirst; the start is then a no-op. network: mainnetdefaults to the public mainnet endpoints:wss://rpc.mainnet.midnight.network/(DEFAULT_NODE_URLS.mainnet, before: the preprod relay) andindexer.mainnet.midnight.network(before:indexer.midnight.network, which does not answer).- Version in
package.json, lock and the compose default tag.
Other 6
Notes
- The indexer supplement pass is paced:
crawler.supplementBlocksPerSecond/NIGHTGATE_CRAWLER_SUPPLEMENT_MAX_BPS(default 2) spaces its requests per block; the batch of 25 per second used to go out at once. The public indexers block the whole host IP with a 403 from their load balancer at roughly 15 per second, and that block also hit the sponsor facades' WebSocket on the hosted box (2026-09-24, after ~1200 blocks). - On a 403 or 429 the pass backs off for one minute, doubling up to fifteen,
instead of retrying every four seconds and keeping the block alive
(
IndexerHttpErrorwith the status,isIndexerRateLimit()insrv/crawler/indexer-supplement.ts); a pass that gets through resets it. - The pass can read its own indexer:
crawler.indexerUrl/NIGHTGATE_CRAWLER_INDEXER_URL(default: the submission side'sNIGHTGATE_INDEXER_HTTP_URL). A private indexer keeps the backfill off the public one; the start log names the host and the rate. stop()wakes a pass that is backing off and ends a pass in flight after the block it is on (cursor recorded);pauseCrawlerand the shutdown no longer wait a backoff out.- Tests: the 500 ms slots at two per second, the 60 s / 120 s / interval ladder across a 403, a 429 and a success, the client's status on a 403, the two env overrides, stop during a backoff and during a pass.
- Version in
package.json, lock and the compose default tag.
- The indexer supplement pass is paced:
Other 8
Notes
- One catch-up at a time:
catchUp()insrv/crawler/Crawler.tsjoins a run in flight instead of starting a twin (catchUpInFlight). The pipeline's second pass and the live gap handler fired within milliseconds of each other once a long catch-up ended (twoCatch-up: 2684838 → 2684909lines on the hosted box), both persisted the same blocks, and the loser'sduplicate key value violates unique constraint "midnight_blocks_hash"latched as a poison block:syncStatus = error, readiness 503, watchdog restart, sponsor facades gone. Twice on 2026-09-24 (05:11 and 05:29 UTC), each time right after the catch-up the previous restart had caused. - A block another writer landed meanwhile is already indexed, not a fault:
persistFromNodereturns the zero-count result when the unique violation names a block that exists (isUniqueViolation()insrv/utils/retry.ts: PostgreSQL, SQLite and HANA messages). A unique violation without the row still propagates and still latches. docker/watchdog.sh, two stages: when readiness reports only the crawler down (database, runtime, initialization fine) it cycles the crawler in place (pauseCrawler+resumeCrawlerthrough the operator API inside the container) and restarts the container only if that has not helped three checks later; any other failing check restarts at once.docs/docker.mdpoints at the script instead of carrying its own copy.resolveSpecVersion: a node answer that disagrees with the sharedLastRuntimeUpgradevalue stays with the block that asked (the one carrying the upgrade); a block that waited on that lookup asks for itself. Two batches fetched at once used to give the upgrade block's version to its predecessor.- ORDER BY on PostgreSQL keeps the NULLS clause for a NOT NULL column
reached through a join and for a column alias of one
(
$orderby=parent/height: the outer join yields NULL); only the query's own columns drop it (stripNullsForNotNullreads the query'sfromandcolumns). docker/watchdog.shdefaults to the compose file's container nameodatano-nightgate; another name goes in the cron line asNIGHTGATE_CONTAINER=<name>, and a containerdocker inspectcannot find is logged instead of silently skipped.- Tests: the join, a live head arriving as the subscription opens, the twin writer against the real unique index, the helper's messages, the parallel runtime lookup, ORDER BY through a join.
- Version in
package.json, lock and the compose default tag.
- One catch-up at a time:
Other 1
Notes
- ORDER BY on PostgreSQL no longer carries a NULLS clause for key and
not nullcolumns.@cap-js/postgresrenders every ordering term asASC NULLS FIRST/DESC NULLS LAST(SQLite's null order), the opposite of a Postgres btree index, so the planner sorted the whole table for anyORDER BY … LIMITon such a column. CAP orders every$topread by the key:Blocks?$top=1wasORDER BY "$b".ID ASC NULLS FIRST LIMIT 1, a parallel seq scan and top-N sort over 2M rows, 0.8 to 7.6 s on the hosted box;Blocks/latest()(height DESC NULLS LAST) 0.9 s althoughng_blocks_heightexists;ContractActions?$top=10.9 s. With the clause gone the primary keys and the existing secondary indexes serve these reads (srv/utils/pg-order-nulls.ts, installed fromsrc/index.tsbeforeensureIndexeswhen the db kind is postgres; rendered through the real driver in the unit test). No new index, no schema change.
- ORDER BY on PostgreSQL no longer carries a NULLS clause for key and
Other 1
Notes
- Agent tokens may call the bound read functions of the indexer entities
without an allowlist entry:
Blocks/latest,byHeight,range,Transactions/byHash,byType,ContractActions/byAddress,history,UnshieldedUtxos/byOwner,unspent,NightBalances/getBalance,getTopHolders(AGENT_ALWAYS_ALLOWED_EVENTS). They read the same public rowsREADalready admits, but a bound function arrives as its own event and the names were not grantable either, so an ODATANO ACCESS key got 403 onBlocks/latest()although the gateway's reference offered the call.
- Agent tokens may call the bound read functions of the indexer entities
without an allowlist entry:
Other 3
Notes
- The crawler no longer asks the node for the runtime version of every block.
state_getRuntimeVersionat a historical hash makes a Substrate node load and compile that block's runtime unless it is one of the two it keeps instantiated (--runtime-cache-size); against the public preprod RPC that was 2 to 8 s per call for every block behind a runtime upgrade, eight of them in each batch frame, so the frame took 17 to 38 s against a 30 s timeout and the catch-up fell from 20 to 0.5 blocks/s at height 1,568,000 (the upgrade to spec 1000000) while every retry looked transient. The batch frame now readsSystem.LastRuntimeUpgradeinstead, a plain storage read at any height, and the node is asked for the version once per distinct value: an upgrade still applies from its first block, a million blocks under one runtime cost one call. The node remains the authority; the decoded value cross-checks it, and an answer that disagrees with the value (the block carrying an upgrade still reports the previous one) is used for that block only, never cached under the value. A block without that storage (pruned or racing node) is asked per block as before. Same for the on-demand path (processBlockByHash). - A batch-frame timeout names the frame (
in a batch frame of N calls), not only the first call in it: the old message blamedchain_getBlockfor a frame that was waiting on something else. - The
Timestamp.Nowstorage key was wrong (its second half hashed to no pallet item), so the node answered null for every block and the block time came from theTimestamp.setinherent: one wasted read per block, and a dependency on the pallet map for a value the state holds directly. The key is now twox128("Timestamp") + twox128("Now"); the inherent stays the fallback for a pruned or racing node. Same value, same column.
- The crawler no longer asks the node for the runtime version of every block.
Other 3
Notes
SyncState.syncProgressreports the share of the CHAIN that is indexed (lastIndexedHeight / chainHeight), not the progress of the current catch-up run. A run starts wherever the cursor left off, so the old figure fell on every restart while the index kept growing: 36 % indexed read as 5.7 %. The column now says which of the two it is, and a chain that is still at genesis reports 100 rather than null.- A node transport fault no longer shuts the server down. CAP ends the process
on any unhandled rejection, so a crawler talking to a slow or incomplete node
could take the submission side with it, and every restart costs each sponsor
facade its warm-up. Faults carrying the node transport's signature are logged
and counted (
getMetrics(),absorbed_transport_faults); everything else reachescds.shutdown, the same end CAP would have brought about, after any listener that was already registered has seen it: an error reporter listens here too and ends nothing, so a captured listener is never taken for a shutdown. A fault qualifies only when the message reads like the node transport AND the stack comes from the crawler or its provider: a defect thrown inside the crawler, or a connection reset on the submission side, still ends the process.NIGHTGATE_CRAWLER_FAULT_GUARD=falserestores the old behaviour. - A prefetched block batch carries a rejection handler from the moment it is
queued. It sits in the queue until the loop reaches it and can fail before
anything awaits it; the queue's own
awaitstill reports the failure.
Other 9
Notes
- The crawler indexes unshielded UTXOs.
Midnight.UnshieldedTokensinSystem.Eventscarries a transaction's consumed and produced outputs, and the crawler already fetched and decoded those records;UnshieldedUtxosandNightBalancesnow fill from them. The Bech32m owner and the DUSTinitialNonceare derived per row and match the Midnight indexer's values. ContractActionscarry theiraddressand record a maintenance update asUPDATE, one row per action the chain reported, so a transaction that only moved tokens no longer produces a spuriousCALLrow. These are the actions that APPLIED; a call in a failed segment is not among them.TransactionscarryledgerTxHash,contractAddress, andsenderAddress/receiverAddress/nightAmountfor an unambiguous one-to-one transfer.txTypefollows the reported events, so deploys, maintenance updates and plain transfers are told apart.TransactionResultsrecordPARTIAL_SUCCESS, which the model had and nothing ever wrote.- Two optional passes trail the indexed tip, both off by default.
crawler.decodePayloads/NIGHTGATE_CRAWLER_DECODE_PAYLOADSdecodes the stored ledger payload into circuit names, identifiers and the zswap and DUST counts, recording the outcome inTransactions.payloadDecode.crawler.indexerSupplement/NIGHTGATE_CRAWLER_INDEXER_SUPPLEMENTfills what a block does not carry from the Midnight indexer:TransactionFees,TransactionSegments,ContractActions.state,ContractBalances, both ledger-event streams andregisteredForDustGeneration. - Both passes keep a cursor on
SyncStateand read it withreorgGenerationin one query, advancing only when neither moved: a rollback to the cursor's own height leaves the cursor alone, so the generation is the signal that a pass's work is gone. Resetting a cursor replays the range. NightBalancescounts NIGHT only; other tokens in the same UTXO set were being added to it.Blocks.stateRootholds the substrate header's state root, which was being written into a column namedledgerParameters. That column now holds the ledger parameters, filled by the supplement and only when they change.- Schema delta.
UnshieldedUtxosis unique on(intentHash, outputIndex): one transaction can carry several intents whose outputs both start at 0. New columns onTransactions,ContractActions,BlocksandSyncState.npx nightgate-schema-deltamigrates SQLite. An index built before this release holds no UTXO rows; a full history needs a re-index.
- The crawler indexes unshielded UTXOs.
Other 3
Notes
- The crawler can index next to the submission side.
crawler.startHeight/NIGHTGATE_CRAWLER_START_HEIGHTbegins at a chosen height instead of genesis: while the index is empty the block below it is indexed as the anchor, the one block allowed to have no parent, and catch-up starts above it. Once the index holds blocks the cursor decides, so a restart resumes where it stopped and a rollback that empties the table seeds again.crawler.maxBlocksPerSecond/NIGHTGATE_CRAWLER_MAX_BPScaps the catch-up rate; persist is the pacer and the fetch queue is bounded, so the cap bounds fetching too. Both are unset by default. - A block that fails deterministically no longer runs on a loop. Catch-up
ended with
syncStatusaterror, the live subscription overwrote it withsynced, and every finalized head re-entered the same block about once per six seconds. The height is latched instead:chainHeightkeeps following the chain, nothing is indexed, and the log andlastErrorname it.pauseCrawler+resumeCrawler,reindexFromHeightor a restart retries it. A transient failure, a node outage above all, is not latched. getReadiness()answers 503 when a check fails. It returned 200 withready: falsein the payload, which orchestrators and the container healthcheck read as healthy. A host that restarts unhealthy containers will now act on a process that is persistently not ready.
- The crawler can index next to the submission side.
Other 1
Notes
- One status surface. The plain
/nightgate/metrics|health|readyroutes,NIGHTGATE_STATUS_TOKEN,NIGHTGATE_STATUS_ROUTES,NIGHTGATE_STATUS_ROUTES_PREFIXand the internal token the image entrypoint generated for its own healthcheck are removed. The read-only probes of the indexer service (getLiveness,getReadiness,getMetrics,getSyncStatus,getHealth) are anonymous at the model level since 0.24.3 and answer the same payloads through CAP; the container HEALTHCHECK now asksgetReadiness().getMetrics()keeps returning the Prometheus text inside the OData envelope. A scraper that used/nightgate/metricsreadsgetMetrics()and unwrapsvalue.
- One status surface. The plain
Other 1
Notes
- Transport auth of the standalone image moves to
@odatano/cap-auth(cds.requires.auth.impl,kind: basic,realm: nightgate). The package runs its basic lane (timing-safe compare, 20 failures per 15 min per client address and user, then 429 withRetry-After), then the registered lanes, then CAP's own strategy forkind; it sends no terminal 401 of its own. NIGHTGATE registers two lanes from the plugin (srv/utils/transport-lanes.ts): the agent token on/api/v1/nightgate(marker principal, the grant hook authenticates, also per$batchpart) and the public verify lane underNIGHTGATE_PUBLIC_VERIFY. A request without credentials is CAP's: the anonymous indexer probes (getLiveness, getReadiness, getMetrics, getSyncStatus, getHealth) answer 200 through the model on the image now (0.24.2 opened them in the model; the image's middleware still refused them before CAP saw the request), everythingauthenticated-usergets CAP'sWWW-Authenticatechallenge. A wrong basic credential never reaches a lane. The model test pins a service-level@requireson every served service; the package's contract table runs against the booted services (test/unit/transport-auth-contract.test.ts).srv/utils/agent-token-auth.tsis gone. Consumer hosts are unaffected (the lanes only act under@odatano/cap-auth).
- Transport auth of the standalone image moves to
Other 2
Notes
NightgateIndexerService.getLiveness()answered 401 without credentials underNODE_ENV=production. CAP authorizes the service before the operation, and a service without a service-level@requiresis implicitlyauthenticated-userin production, so the function's ownanywas never reached (seen live behind ODATANO ACCESS: the gateway's credential-free probe had to fall back to operator auth). The service is now@requires: 'any'with the requirement on every element. The read-only probes the model always meant for K8s and Prometheus (getLiveness, getReadiness, getMetrics, getSyncStatus, getHealth) are public now, as their test claimed; SyncState, ReorgLog, getReorgHistory, getRuntimeInfo and getWorkerStatus stayauthenticated-user, pause/resume/reindex stayadmin. Behind api.nightgate.dev the gateway still requires a key for/api/v1/indexer/*. A model test pins the layout (test/unit/service-auth-annotations.test.ts).eslintis pinned exactly (10.11.0).@sap/cds-dkbundles@eslint/js, whoseeslintpeer range npm re-resolves against the registry duringnpm ci; with a caret range every new eslint release made the lockfile look out of sync and failed CI.
Other 4
Notes
- Every submit goes out on its own phased node client (connect / request /
watch budgets, indexer lookup on an ambiguous outcome). Bound submits
(deploys, calls, batches, sends, dust registration, bound sponsoring) used
the wallet SDK's
submitTransaction, whose promise settles only when the node closes the shared socket: a1010reject held the worker for minutes, and the sponsor pool read as unsynced meanwhile. The wallet's pending bookkeeping is kept (pend before the send, revert on failure);NIGHTGATE_SUBMIT_TRANSPORT_RETRIESnow also covers a failed connect phase;NIGHTGATE_SPONSOR_WAITapplies to all submits (default InBlock, the indexer confirmer records finality on the job). A landed transaction whose call did not apply fails asTxFailedErrorwith the block height on every path, afterInBlockand afterFinalizedalike. A submit whose every attempt failed before a send restores the dust snapshot like a pre-mempool reject (the SDK revert frees the pending marker, not the note). Once a send may have reached the node and the indexer does not show it, every later failure of the same submit isambiguous(SubmitAmbiguous, identifier kept, no rebuild, no dust restore), whatever the later attempt answered. - Sync gate: a failed read of the ledger-event stream tip (one-shot indexer
subscription) reuses the last read for
NIGHTGATE_STREAM_TIP_GRACE_MS(default 3 min,0= off) instead of reporting the tip unknown, which failed the gate for that tick and took a sponsor out of the pool for a minute at a time. - Compose:
init: trueon thenightgateservice (tini reaps the zombies a killed health check leaves behind node as PID 1). - Docs: anchored roots, schema ids, set roots and claim keys are bound to
the artifact generation (
transientHash);payloadHashis not. Keep theopening, a vault on a new compiler generation is re-anchored from it.
- Every submit goes out on its own phased node client (connect / request /
watch budgets, indexer lookup on an ambiguous outcome). Bound submits
(deploys, calls, batches, sends, dust registration, bound sponsoring) used
the wallet SDK's
Other 14
Notes
Vault lineage 4 (both widths),
@odatano/nightgate-tx0.6.0. BREAKING: redeploy and re-anchor. Schema delta:BackgroundJobs.grantId,Documents.attesterId,DisclosureGrants.attesterId/changedAtHeight,PredicateAttestations.attesterId/attesterIdB(nightgate-schema-delta).- Records keyed by
recordKey(attesterId, payloadHash); commit-reveal, sequence and epoch removed; an attest is one transaction. retract(mode, key): own record (0) or expired claim (1). Claims carryvalid_until(max five years), extend-only on re-proof.- Claim keys embed record key, content root and schema id.
registerDocument/bindDocumentreplace the passport circuits; a re-registered id drops a foreign binding. Constructor(registrar, recovery); recovery re-points the registrar (mode 3) or itself (mode 4).- Actions:
anchorDocumentreturnsattesterId;issue*takeattesterIdandvalidUntil(NIGHTGATE_CLAIM_LIFETIME_S);verifyAttestationStatebyattesterId+payloadHashordocumentId, reportsrecordKey,bindingRegistered;registerPassportmode0-4;deployContractrecoveryId; newretractAttestation,purgeExpired. verifyDocument/verifyPredicateAttestation:verifiedneeds the live read;includedandstateCheckedreported separately.- Browser / txbuilder: proof helpers take
recordKeyandvalidUntil;prepareRegisterDocument,prepareBindDocument,prepareRetract; vaultconstructorArgs[registrarId, recoveryId];keys/manifest.jsoncovers verifier keys and zkir. submitContractCallBatchmerkleProofcarriesfieldSalt, acceptsdocPair;thresholdat most 2^63 - 1.- Disclosure projection ordered by block height; confirmation writes never
overwrite a newer row; a failed post-submit reindex retries as a
reindexDisclosuresjob (NIGHTGATE_DISCLOSURE_REINDEX_RETRY_MS). - Jobs re-read their agent grant at execution (
AGENT_GRANT_REVOKED,AGENT_GRANT_SCOPE). updateAgentGrant,rotateAgentGrantToken,getGrantUsage(NIGHTGATE_GRANT_ADMIN_RATE_LIMIT); public verify lane/api/v1/verify(NIGHTGATE_PUBLIC_VERIFY,NIGHTGATE_PUBLIC_VERIFY_RATE_LIMIT).- Submit: late intent ack fails the job (
SubmitIntentTimeout,NIGHTGATE_SUBMIT_INTENT_ACK_TIMEOUT_MS); unbound dust intent takes the lowest free segment;1010/104rebuilt (NIGHTGATE_STALE_TRANSCRIPT_RETRIES). - Wallet: rejected snapshot replay resets the sub-wallet
(
NIGHTGATE_SNAPSHOT_REPLAY_RESET_MS);sync-statelog (NIGHTGATE_SYNC_STATE_LOG_MS);getSponsorPoolStatus.caughtUpis the sync gate; pool jobs prefer members at the gate. /metricsreports the database pool;qsoverridden to 6.16.0.
- Records keyed by
Other 11
Notes
Authorization, key handling and runtime gating. No circuit change; schema delta: two nullable columns (
DisclosureGrants.pendingLevel,Documents.sessionId), added bycds deployat boot. Storedv2envelopes stay readable; runnightgate-rewrap-keysto bind them.- A disclosure level changes only once the chain took it.
grantDisclosureon an existing grant used to write the requested level into the row before the session check and before any confirmation, and the off-chain read ACL trusts that row. The row is now written after the session check, an existing row keeps its confirmedleveland carries the request aspendingLevel(new nullable column, additive); the executor and the reconciliation finalizer move it intolevelon inclusion and drop it when the chain refuses. A refused caller leaves the row untouched. - Write actions answer 503 while the runtime is down. Initialisation
failed or not completed: every action that needs the wallet worker, node or
proof server (
connectWallet, sends, deploys, submits, grants) is refused withRUNTIME_UNAVAILABLE,Retry-Afterand a message that survives production sanitising and names no internals (the startup error stays in the log and the admin status), instead of creating half-backed state. Reads, readiness and the compute-only actions (prepareDocumentProof,prepareMembershipSet,deriveWalletInfo,getJobStatus, grant admin) stay reachable. - Agent-token reads stop at the session. A token could list every
Documentsrow of its operator,storageRefincluded, because entity reads were only owner-scoped. Tokens now read an explicit entity set (chain projections, plusWalletSessions,PendingSubmissions,Documentsnarrowed to the grant's session and the grant's ownAgentGrantsrow); everything else answers 403. Schema delta: one nullableDocuments.sessionIdcolumn, filled byanchorDocument; rows from before the column stay owner-readable and never token-readable. - Grant lists bound every action.
allowedContracts/allowedCircuitsnow refuse (403) any token action outside them, not only sponsored calls; an empty list is still no restriction. The circuits are derived from the action itself (grantDisclosurerunsgrantDisclosure,anchorDocumentcommits withattestGuarded, orattestwithguarded: false, theissue*actions their proof circuits plusanchorContentRoot) and from batchcalls, not only from acircuitfield; an action whose circuits cannot be derived is refused while a circuit list is set. The execution-time policy treats a grant pastvalidUntillike a revoked one, so an expired grant sponsors no queued job. - Tier gates resolve the role themselves. The SDK's
Disclosed/Authoritybefore-handlers readreq.disclosureRolethat the*hook attaches; CAP starts both at once, so a reader could be refused before the role was there. The gate now attaches the role itself when it is missing, and both dispatch orders give the same answer. - Stored envelopes are bound to their row. New ciphertexts are
v3envelopes whose AAD carries key id, purpose and row id (session id, job id, account id); a value copied to another row or column does not decrypt.v2stays readable;nightgate-rewrap-keysrewrites it. - The viewing-key seal of the account key needs the ring too. The
vk1seal is now wrapped in a ring envelope: a database copy plus a viewing key opens no data key, private state or contract signing key withoutENCRYPTION_KEY. Bare seals are wrapped on first use and by the rewrap tool. A ring key that leaves without a rewrap can no longer be replaced by the viewing key; run the rewrap first. - Signing-key custody. Admin action
exportContractSigningKey(sessionId, contractAddress, password)returns one contract's maintenance authority as themidnight-signing-key-exportenvelopeimportSigningKeysrestores; procedure in docs/operations.md. Production refuses an encryption secret shorter than 32 characters instead of warning. - The real-circuit contract checks are Vitest suites now:
test/integration/attestation-vault.test.ts(76 scenarios) andattestation-vault-32.test.ts(18), one test per guard, each describe on a fresh contract state;npm testruns them,npm run test:contractruns just those. The sequential scriptsscripts/integration-test-attestation-vault[-32].mjsand their npm aliases are gone;check:releaseno longer needs the compiled srv twins for them. - SDK client survives a closed keep-alive socket, without doubling a write.
A caller that proves locally between two requests came back to a pooled
socket the server had already closed and got
fetch failed/ECONNRESET.connect()now retries such a request once on a fresh connection, but only when a second delivery cannot create a second effect: GETs, and POSTs that carry anidempotencyKeythe server dedupes on. A write without a key surfaces the error after one attempt; passidempotencyKeyon submit actions to get the retry. - Wording. Comments and docs describe the vault in terms of documents,
canonical JSON and external identifiers; the circuit and action names
(
registerPassport,bindPassport,passportId) are unchanged API.
- A disclosure level changes only once the chain took it.
Other 6
Notes
A production hang and a lost broadcast. No schema or circuit change.
- A lost broadcast ends the job. A sponsored transaction the node never
included (four in ~2100 sponsorings: no reject, no status, never on the
indexer) parked its job in
reconciliation_requiredforever. Every submit intent now records the transaction'sttlon the attempt row; once the indexer tip is past it plusNIGHTGATE_BROADCAST_EXPIRY_MARGIN_MS(5 min) and the indexer has no record, the job endsfailed / BROADCAST_NOT_INCLUDED,chainStatus: dropped. Only absence counts (an indexed transaction without a confirmable result stays parked), the tip comes from the same indexer answer as the absence (the older one when identifier and hash are both tried), and without a tip there is no verdict. A lost sponsored deploy refunds its deploy reservation; a workflow parent whose child was lost endsfailed / CHILD_FAILED. Rows from before this release use the submit time plus one hour. BROADCAST_UNCONFIRMEDwhile parked. The ambiguous submit outcome has its own park code;EXTERNAL_EXECUTION_FAILEDstays for a failure after the broadcast.- The submit says which phase died. The dedicated-client submit
(unbound sponsoring) drives the node client's event stream itself
(
srv/midnight/worker/phased-submit.ts) with one budget per phase: connect (NIGHTGATE_SUBMIT_CONNECT_TIMEOUT_MS, 20 s; nothing sent, retried once on a fresh client, then a clean pre-inclusion failure), request (NIGHTGATE_SUBMIT_REQUEST_TIMEOUT_MS, 30 s, until the node's first status; ambiguousno-reply) and watch (NIGHTGATE_SUBMIT_WATCH_TIMEOUT_MS, 75 s, until InBlock; ambiguous). Every status and socket event is logged with its offset; a timed-out attempt keeps listening forNIGHTGATE_SUBMIT_LATE_GRACE_MS(5 min) and logs a late reject or InBlock under the identifier; closing an abandoned client is bounded.@midnightntwrk/wallet-sdk-node-clientandeffectare direct dependencies now. - No concurrent recompilation in the container. A live 0.23.1 server
stopped answering for an hour: a lock-order deadlock inside V8 (node
22.23.2) between the main thread at a GC safepoint and a background
TurboFan compile holding the same transition-array lock. The entrypoint
starts node with
--no-concurrent-recompilation;NIGHTGATE_NODE_FLAGSoverrides the flag list (V8 flags are refused inNODE_OPTIONS). docs/docker.mdcarries a cron watchdog on the healthcheck: Docker never restarts an unhealthy container by itself.- Image tag
0.23.3.
- A lost broadcast ends the job. A sponsored transaction the node never
included (four in ~2100 sponsorings: no reject, no status, never on the
indexer) parked its job in
Other 3
Notes
@odatano/nightgate-tx0.5.1, the fix for a 0.5.0 packaging break.- nightgate-tx 0.5.1. Every in-process (wasm) build of 0.5.0 failed on
load with
Cannot find module '../utils/config': the shipped proof provider and the batch scope imported the server's config table statically, and the table is not in the package (server proving was unaffected). Both now read throughsrv/midnight/runtime-config: the config table inside the server, the environment in the package.check:slimwalks every relative require of a shipped runtime file and refuses one that resolves outside the package; the real-install probe loads the proof provider from the clean install and checks the environment fallback. - The package README documents the ledger-v8 override a fresh consumer
install needs: the wallet SDK declares
^8.1.x, a second copy (8.1.1) next to the pinned 8.1.0 fails the dust wallet withexpected instance of DustParameters; pin"@midnight-ntwrk/ledger-v8": "8.1.0"inoverrides. - Image tag
0.23.2; no server behaviour change.
- nightgate-tx 0.5.1. Every in-process (wasm) build of 0.5.0 failed on
load with
Breaking 1, Other 2
Notes
@odatano/nightgate-tx0.5.0, the caller-side companion of 0.23.0. The server release carried the generated package at 0.4.5, which builds lineage-2 vault calls.- nightgate-tx 0.5.0 (BREAKING). Ships the lineage-3 vault modules
(
attestation-vault,attestation-vault-32);prepareAttestCommittakesexpiresAtandprepareAttestRevealemits the five-argumentattestGuarded, so a 0.4.x caller fails against a redeployed vault. The witness builder and hex codec are the sharedsrc/browsersources; the zk-asset fetch accepts gzip-encoded responses;prepublishOnlyrefuses a stale generated tree. - Image tag
0.23.1; no server code change.
- nightgate-tx 0.5.0 (BREAKING). Ships the lineage-3 vault modules
(
Breaking 6, Security 4, Other 16
Notes
Hardening release across the whole server: authentication, submission bookkeeping, encryption at rest, crawler correctness, worker structure, configuration and packaging. Schema delta:
ContractActions.addressnullable, inclusion columns onBackgroundJobsandPendingSubmissions,SyncState.reorgGeneration, newAccountKeystable,keySchememarker on the private-state tables;nightgate-schema-deltaapplies all of it, the image redeploys. Breaking items first; every deployed attestation vault must be redeployed and its documents re-anchored.Breaking
- Vault lineage 3.
attestGuarded(mode, payload, meta, nonce, expires_at): a commitment is bound to its committer, expires at block time (7-day cap), a fresh reveal inherits the commitment's sequence and a revealed attestation is final.anchorDocumentis guarded by default (guarded: falsefor public hashes);commitDocumentAnchorand browserprepareAttestCommittakeexpiresAt. Both widths recompiled. - Prover keys leave the npm tarball (0.72 MB packed, was 86.9 MB). A
missing
keys/<circuit>.proveris fetched fromNIGHTGATE_ZK_ASSET_URL(default: the release's GitHub tree for shipped contracts) and verified againstkeys/manifest.json;nightgate-fetch-keyspre-fetches for offline installs. The artifact digest covers verifier keys, zkir, module and manifest; 0.22 digests still match aslegacy. - Nothing is derived from the extrinsic envelope. No contract address
from the extrinsic hash, no NIGHT transfer, UTXO or balance from a signed
extrinsic's call args. Those
Transactionscolumns andContractActions.addressare null until the ledger payload is decoded;ContractStatisticscounts per action type. The delta clears old values. - Canonical JSON follows RFC 8785: integer-like keys sort as strings.
/contract-manifestemits relative URLs unlessNIGHTGATE_ZK_CONFIG_PUBLIC_URLis set; a dApp on another origin passesmanifestUrltocreateNightgateConnectorProvidersor usesresolveManifestUrl.- Removed:
probeCrossServerSponsor(action, executor, worker handler),crawlerlessChainConfirm(the confirmer is always on), the lease fields ofgetJobStatus. Node 22.12 or newer.
Security
$batchparts are authenticated from the envelope token; the transport principal without a token is rejected. Owner-scoped entity reads under a token return the caller's rows.- The agent token never reaches the request log: CAP's JSON format prints
every header and masked only its defaults.
src/cap-log-mask-boot.tsaddsx-agent-tokentolog.mask_headersas the first import of every entry point (the formatter freezes the list on a logger's first use); the image config states it too. Rotate tokens issued earlier if the container logs were readable by anyone but the operator. - Rate limits are keyed by grant, then user, then address, 64 keys per
principal with LRU eviction; new limits on the sponsor actions (120/h)
and
buildSponsorable(30/h). Failed basic-auth attempts are throttled per address (20 per 15 min, then 429 withRetry-After). - Sponsor policy is resolved when the job runs (
AGENT_GRANT_REVOKEDfor a revoked grant); an unpinned grant cannot name a sponsor; a wallet facade stays warm only for the same user's sessions; adminWalletSessionsandDisclosureRolesare read-only;prepareDocumentProofresolves own properties only and accepts decimal numeric strings only.
Encryption at rest
- Key ring:
ENCRYPTION_KEYS=id=secret,...+ENCRYPTION_KEY_ACTIVE(ENCRYPTION_KEYalone = id1). Ciphertexts are v2 envelopes (per-row DEK under an HKDF-stretched KEK, key id in the AAD); legacyiv:tag:datastill reads. The dev fallback key is random per process. - Account data keys (
AccountKeys): private states, contract signing keys and sync-state blobs are encrypted under a per-account DEK sealed under the ring and under the viewing-key password.nightgate-rewrap-keysrotates every account without a viewing key and exits 1 while legacy rows remain; the boot preflight refuses an unknown key id. Cached DEKs are zeroed on disconnect and shutdown, and a resolution in flight at eviction time does not repopulate the cache.
Submission
- Every submit path announces the transaction identifier before it
broadcasts and persists it as the job's
txHashfirst; rejected attempts are closed, refunded and taken off the job in one transaction; restart recovery fails hash-lessexternal_executionrows plainly (PROCESS_RESTART_BEFORE_BROADCAST). - Chain evidence is the indexer confirmer's block:
chainBlockHeight,chainBlockHash,indexerTxHashon job and attempt row, written in one transaction and never without a height. Every commit is reorg-safe: each rollback incrementsSyncState.reorgGenerationas its first write (one atomic+ 1that holds the row lock for the whole rollback), a confirmer captures the generation before its lookup and refuses the commit when it moved. A reorg rollback andreindexFromHeightrevert every job and attempt confirmed at or above the fork by that height, chain-failed attempts included. A failure the worker proves directly goes through the same lookup or parks asCHAIN_EXECUTION_FAILED_UNCONFIRMED. - Submit failures are classified once in the worker (
pre-mempool-reject,dust-race,transport,ambiguous,landed-not-applied,policy,causality,internal) and travel the RPC as data (WorkerSubmitError); both sponsor channels share one decision table (NIGHTGATE_SPONSOR_DUST_RETRIES). A wait that ended without a reply is ambiguous, never resent. - Unbound sponsoring picks the free dust backing with the most headroom, values the notes at the spend's block time and refuses a backing whose fresh note no longer covers the fee before proving (three re-selects).
- Job kinds declare their traits in one table (
srv/submission/job-kinds.ts); the heavy, workflow-parent, identifier-keyed and session-bound sets derive from it. Expired leases are reclaimed (NIGHTGATE_JOB_LEASE_TTL_MS), the poller claims rows up to free capacity, the equality/membership/integrity/ diff workflow parents reconcile. - Worker rotation drains submitting calls only, bounded by
NIGHTGATE_WORKER_DRAIN_MAX_MS; rotation and shutdown flush every facade with an acked save; reads cut by a rotation are repeated once (WORKER_ROTATED); sync waits and private-state round trips are bounded.
Crawler
Transactions.rawis a binary value (PostgreSQL-safe; the delta clears lossy text values,reindexFromHeight(0)restores them). Block timestamps come from storage or theTimestamp.setinherent, never the wall clock. The runtime version rides per block; registry and pallet map are cached perspecVersionand a block is classified with its own runtime's map. A missing or invalid runtime version, an unreadable metadata or one that lists no pallets fails the block instead of guessing.- The node provider never stops reconnecting,
stop()waits for the in-flight persist, a failed header lookup during the fork search propagates, secondary indexes are created at startup, lock contention is recognised by SQLSTATE (40001, 40P01, 55P03, 57014) with a shared retry.
Structure
- The wallet worker is eleven modules under
srv/midnight/worker/;wallet-worker.tsis the composition root. Artifact snapshots live per install (<base>/<install>/<digest>with holder files). - One typed config table (
srv/utils/config-table.ts, 80 keys): uniform parsing, every key settable viacds.requires.nightgate, the worker reads the resolved snapshot fromworkerData,docs/reference.mdis generated (npm run config:table) and pinned by a test. - One witness builder and one hex codec (
src/browser/witnesses.mjs,hex.mjs) for server, worker and browser;npm run check:vault-paritykeeps the 16- and 32-slot sources in step;slotWidthis 16 or 32. Main-thread artifact readers are generation-pinned; a job resolve compares the memoised digest instead of re-hashing the prover keys.
Packaging and operations
- PostgreSQL integration lane (
npm run integration:postgres), a CI job and part of the tag gate (check:release:full).nightgate-schema-deltaneeds no cds-dk.check:exportsverifies everybin, every manifest circuit's verifier/zkir/bzkir in the tarball, and that packed build output has a tracked source. - Image runs as
node;.dockerignoreexcludespackages/,deploy/,.github/;deploy/contabo/removed; the compose tag is pinned to the package version by a test;packages/nightgate-txpublishes only from a fresh generated tree; the txbuilder's asset fetch accepts gzip responses. getRuntimeInfo().apiVersionand a written 0.x API rule;estimateSendNightFeetakestokenTypeHex;mintShieldedTestTokenruns under the heavy cap; bins are linted.
- Vault lineage 3.
Other 5
Notes
Midnight SDK line moved to midnight-js 4.1.1 / compact-js 2.5.1 / ledger-v8 8.1.0 (exact pins) in
@odatano/nightgateand@odatano/nightgate-tx0.4.5. No schema change, no circuit change, no action change.- One copy of every Midnight package. midnight-js 4.1.1 routes all its
ledger, compact-js, compact-runtime, onchain-runtime and platform-js
imports through the new
@midnight-ntwrk/midnight-js-protocol, which pins ledger-v8 8.1.0, compact-js 2.5.1, compact-runtime 0.16.0 exactly. Both packages now pin the same set exactly (midnight-js 4.1.1, compact-js 2.5.1, ledger-v8 8.1.0, compact-runtime 0.16.0, zkir-v2 2.1.0); a consumer on that line dedupes without overrides. Two copies of a wasm-bearing package reject each other's objects with messages likeexpected instance of ContractMaintenanceAuthority. compact-js 2.5.3 is NOT the target: it depends on a ledger-v9 alpha. The 0.22.1 tree itself carried a second compact-runtime (0.15.0 nested under compact-js 2.5.0); gone. - Zswap offers follow the transcript segment (upstream, midnight-js
contracts 4.1.1): shielded outputs, inputs and transients of a contract
call are placed in the guaranteed or fallible segment according to where
the contract's transcript claims them, instead of always in segment 0.
The sponsor's offer check already inspected both
guaranteedOfferandfallibleOffer; nothing to change on the NIGHTGATE side. - LevelDB private-state backend password (dev-only opt-in
privateStateBackend: 'level'): midnight-js 4.1.1 validates the storage password (16+ characters, three character classes, no runs, no sequences); the derived 64-char hex has two classes.levelStoragePasswordre-encodes it deterministically for the level provider only (byte pairs joined by-, fixed suffix). A level store written before 0.22.2 does not open with the new key; the default CAP-DB backend is unaffected. - Upstream additions worth knowing: the indexer provider moved to Apollo
client 4 and throws typed
IndexerQueryError/IndexerDataError/IndexerSubscriptionDataErrorinstead of bareErrors; midnight-js warns on the console when a proof-server or indexer URL useshttp:/ws:to a non-loopback host (the compose proof server athttp://proof-server:6300logs it once per provider bundle);NodeZkConfigProviderrejects circuit ids outside[a-zA-Z0-9._-](the standard-circuit lookups such asmidnight/zswap/spendkeep failing over to the wallet SDK's key material as before). - Sizing note for in-process wasm proving, measured on the vault's
proveDocumentComparison: k=17 (38.5 MB prover key) peaks at 1.82 GB RSS, k=18 (72.9 MB) at 3.45 GB; ~47x the prover key, doubling per k. The JS heap stays around 50 MB, the rest is wasm linear memory, which never shrinks (a second prove in the same process adds ~46 MB). Two proves in one process serialize and peak at the maximum, not the sum. UnderprovingMode: 'server'the client still loads the prover key (it travels in every/proverequest) but not the proving working set. Documented indocs/txbuilder.md.
- One copy of every Midnight package. midnight-js 4.1.1 routes all its
ledger, compact-js, compact-runtime, onchain-runtime and platform-js
imports through the new
Other 3
Notes
Single-call
beforehooks and self-funded submission, under@odatano/nightgate/txbuilderand@odatano/nightgate-tx0.4.4 alike. No server behavior change, no schema change, no circuit change.beforehooks run for single calls:buildSponsorable({ call })(and a one-entrycalls) now invokes the call'sbeforehook immediately before proving, exactly like a batch entry. Previously the hook was batch-only: a single call proved with the witnesses as they stood, and a batch split into single-call transactions silently dropped its hooks; an unarmed witness holder then proves value 0 with a zero salt, which on an insert-style circuit lands as a commitment nobody can open.- Self-funded submission exports on
/txbuilder, for callers that pay their own dust and submit to the node themselves:submitFinalized(extrinsic encoding over HTTP via@polkadot/api, a new optional peer dependency; submit over a one-shot WebSocket, the node's HTTP gateway 403s bodies over ~14 KB) plussubmitExtrinsic,deserializeTransaction,txIdentifiers,probeLanded(confirm by identifier, reportsapplied: falsefor in-a-block-but-call-failed),classifyNodeReject(170/171/196 stale dust proof, 138/173 funds, 219-224 sequencing, 117 malformed),isPreMempoolReject,isTransportFailure(probe, then resend the SAME bytes),isAlreadyImported(1013 after a resend = the first send reached the pool, confirm by identifier instead of failing),waitLanded(probeLandedin a bounded loop; ANY refused resend can mean the first send landed while the indexer lags, so wait for the identifier before trusting the reject) andwithDustGuard(dust sub-wallet snapshot before a build, restore on a pre-mempool reject).probeLandedreads a missing transaction result, an HTTP failure or a GraphQL error as unknown (null), never as applied; a socket close before the submit reply fails immediately as transport instead of waiting out the timeout. Docs:docs/txbuilder.md, "Self-funded submission"; the packaged README carries the same flow. - Image chores:
.dockerignoremirrors the.gitignorerules for DB backups, wallet-state dumps and env files, recursively (**/.env*with an.env.exampleexception; dockerignore patterns are rooted, so*.logand.env.*never covered subdirectories and a localCOPY . .build shipped operational data); compose default image tag 0.22.1.
Other 9
Notes
Custom-token balances, struct circuit arguments, proof request timeout, same-transaction resend, contract artifacts outside the package, sponsored zswap offers of a contract's own token. Additive schema migration (one nullable
AgentGrantscolumn). No circuit change.@odatano/nightgate-tx0.4.3.getWalletBalance.shieldedTokens[]: every shielded token type other than NIGHT with a non-zero balance,{ tokenType, amount }(raw 64-hex type, atoms). Pairs withderiveTokenTypeandsendNight(tokenTypeHex); a contract-minted token was spendable but invisible.- Struct circuit arguments on
submitContractCall/submitContractCallBatch: acontract-info.jsonStructparameter (ShieldedCoinInfo,QualifiedShieldedCoinInfo, own structs) is coerced field by field with the field's own type, tagged or introspected, nested structs included; missing field / non-object / field errors keep theargs[i]prefix.Vector/Maybe/Eitherstill pass through. proofTimeoutMs(cds) /NIGHTGATE_PROOF_TIMEOUT_MS: HTTP timeout of one proof request in server proving mode, default 300000 (midnight-js' default), pinned into the env at init like the proving mode. A proof past the timeout failed the job and midnight-js re-requested it up to three times. The proof server'sMIDNIGHT_PROOF_SERVER_JOB_TIMEOUTgoes with it.- Same-transaction resend on a transport failure: when the SEND of a
finalized transaction fails (websocket closed at submit,
1000 Normal Closure,ECONNRESET, no reply), the worker asks the indexer for the identifier, then hands the SAME transaction back tofacade.submitTransaction(which re-pends the spends):NIGHTGATE_SUBMIT_TRANSPORT_RETRIES(2),_BACKOFF_MS(5 s),NIGHTGATE_SUBMIT_LANDED_PROBE_MS(30 s). A landed transaction is reported as submitted only with ledger resultSUCCESS; in a block but not applied (FAILURE/PARTIAL_SUCCESS) fails asTxFailed(fee spent, rebuild). Any refused resend (validity reject,1013 Already Imported) is checked against the indexer before the refusal propagates. Transport = socket closed/reset/refused, submit timeouts (TimeoutError,timed out,no reply). Every bound submit (deploy/call/batch, sends, dust registration, bound sponsoring). Before: the job failed and a re-run rebuilt and re-proved the call (live: a 13 min proof twice). Node rejects are never resent.classifySubmissionErrorcounts the transport lines asNetworkOrTimeout. - Contract artifacts outside the package: the
registerContractvalidation probe imports the artifact from a disposable copy next to anode_moduleslink to NIGHTGATE's runtime (as the worker's snapshots do), soNIGHTGATE_CONTRACTS_DIRpoints at a consumer's own directory without a node_modules there. Documented as the supported layout. - Sponsored zswap offers,
allowedTokenTypes: the sponsor shape check refused every non-empty offer, so a token contract'smint(minted coin to the caller) orburn/receiveShielded(caller's coin into the contract) could never be sponsored although no sponsor value moves. FloorNIGHTGATE_SPONSOR_ALLOWED_TOKEN_TYPES/ policy-fileallowedTokenTypes, grantcreateAgentGrant(..., allowedTokenTypes)(new nullable column, effective = floor ∩ grant, the floor must open it; empty = no offers, unchanged). With a list, an offer passes iff every net change (deltas, public per raw token type) is on a listed type and never NIGHT, every contract-owned coin belongs to a sponsorable contract, and a net change exists or a coin in the offer is owned by a sponsorable contract (the ledger drops zero deltas: a zero-net offer says nothing and is refused, except when the contract itself received or spent a coin in it, which is what a burn looks like: user input, contract transient, burn-address output, net zero). User outputs are commitments, so a transfer of a listed type between users riding along is accepted by design. Entries are raw 64-hex types (deriveTokenType); an invalid floor entry fails closed (503). - txbuilder
proofTimeoutMs(@odatano/nightgate-tx0.4.3):createTxBuilder({ provingMode: 'server', proofServerUrl, proofTimeoutMs })passes the HTTP timeout of one proof request to the SDK (default 5 min, the same ceiling as the server's). The TTL is stamped after proving. - Packaging:
filesnames the four shipped contracts explicitly (counter,attestation-vault,attestation-vault-32,shielded-token) instead ofcontracts/**, so a foreign directory undercontracts/never reaches the tarball;.dockerignoredropsscratch/and those directories from local image builds. Compose: image tag 0.22.0,NIGHTGATE_PROOF_TIMEOUT_MSpassed to the server,MIDNIGHT_PROOF_SERVER_JOB_TIMEOUT(default 600) to the proof-server container, both in.env.example.proofTimeoutMsin the plugin'scds.schema;shieldedTokenson the client/token-ops return types; txbuilderproofProviderConfig()exported. - Docs:
docs/reference.md(config + env),docs/actions.md(struct arguments,shieldedTokens, sponsored offers),docs/operations.md(upgrade note, three troubleshooting entries),docs/txbuilder.md.
Other 3
Notes
Stage-grouped segment order for independent batch calls. No schema change, no SDK change, no circuit change.
@odatano/nightgate-tx0.4.2.independentCallsonsubmitContractCallBatch, the worker's batch op and txbuilderbuildSponsorable({ calls }): the calls pastorderedPrefixshare no state and are grouped by execution stage before proving (guaranteed-only first, call order within a group).issueFieldPredicateAttestationBatchsets it itself, with its optional in-batchanchorContentRootpinned first. Why: a proof cart is a set of claims with distinct keys, and on a grown vaultpartitionTranscriptsputs the same circuit in different stages for different keys (measured on demo.zkpassport.eu, vault 4a8893cf:proveFieldPredicate5.86G guaranteed vs 6.05G fallible, budget in between), so call order alone failed the causality pre-check on about every second cart while a valid order existed. Dependent batches leave the flag unset and keep call order, including the fail-fast.- Structured refusal. The pre-check throws
BatchCausalityError(code: 'BatchCausalityViolation',calls: [{ name, segId, stages }]in apply order) and appends the same list to the message (Stages in apply order: ...); the txbuilder restorescallsfrom the message when the SDK wrapper drops the typed error. orderBatchSegments/withOrderedBatchSegments/runBatchInScopetake{ independentCalls, orderedPrefix };batchCallStages(tx)exported.
Other 4
Notes
txbuilder: wallet sync cost, socket leak, identity. No schema change, no SDK change, no circuit change.
@odatano/nightgate-tx0.4.1.close()stopped nothing. It calledfacade.close?.(); the facade hasstop(), so the call was a no-op and every builder's wallet sync (three indexer streams, from genesis) ran until the process exited. Nowfacade.stop()plus the public-data provider's sockets:indexerPublicDataProviderhas no close, so the builder hands it a trackingwssubclass (trackingWebSocket) and terminates what it opened. Measured (attest, bind:false, fresh seed, wasm): idle CPU with the builder open 89-101 %, afterclose()93 % before / 0 % now; indexer sockets 1-2 before / 0 after.walletSync: falseoncreateTxBuilderskipsfacade.start(). A call that moves no value needs no wallet state to build, prove and sign (balancing returns the tx untouched, signing is the keystore alone); a value-moving call fails at balancing. Measured against the default: build 27.7 s vs 24.5 s (proving dominates), event-loop lag p50 7 ms vs 108 ms during the build, idle CPU 2 % vs 101 %, identical 5284 bytes. Sponsored on preprod from a never-synced seed: 006f31fc4a5831e05dddf3d2598912ff0f7b2fdc4c86f4482646b1a3d87fbf9b8e (66 s incl. dust proof in wasm; verifyAttestationState confirms the caller attester id).deriveIdentity({ seedHex, networkId?, accountIndex?, attestationSecret? }):attesterId, attestation secret and NIGHT address without a builder or the network (~150 ms; was a fullcreateTxBuilder). Exported from@odatano/nightgate/txbuilderand thenightgate-txroot.- Docs: everything the builder does runs on the calling thread; host it in
a
worker_threadsworker (docs/txbuilder.md, slim README, example).
Other 3
Notes
Graceful container stop. No schema change, no SDK change.
- Entrypoint runs
exec node /app/node_modules/@sap/cds/bin/serve.js: node is PID 1 and receives SIGTERM, so cds runs its shutdown hooks (wallet state flush, worker exit).exec npx cds-serveput npm -> sh -> node in between andsh -cdoes not forward the signal: everydocker stopended in SIGKILL after 10 s ("Container failed to exit within 10s of kill" in the docker journal), and one recreate raced on the kill and left the new container inCreated(api.nightgate.dev, ~10 min outage, 2026-08-28 12:54 UTC). - Compose
nightgateservice:stop_grace_period: 90s. test/unit/docker-cds-config.test.tspins the entrypoint's exec form and LF line endings.
- Entrypoint runs
Other 3
Notes
Worker GC load from the save tick. No schema change, no SDK change.
- Save tick 60 s, configurable.
NIGHTGATE_SAVE_INTERVAL_MS(default 60000, min 10000; was a fixed 30 s). Measured on the hosted pool withprofileWorker(0.21.5, three warm facades): 63 scavenges of ~180 ms in a 20 s window, GC 42 % of the worker,DustWallet.serializethe only hot path; the dust blob changes with practically every block, so each tick re-serialized and pushed multi-MB strings per facade. - Worker young generation 128 MB.
NIGHTGATE_WORKER_YOUNG_GEN_MB(default 128,0= V8 default 16 MB, 16..2048) setsresourceLimits.maxYoungGenerationSizeMbon the wallet worker; the old-generation limit still comes from NODE_OPTIONS (verified: a worker with only the young size set keeps the inheritedheap_size_limit). - Hosted box: the Contabo image's hourly
drop_cachescron was found and disabled (operations note); it explains the hourly IO stalls behind the 0.21.3 pool incident.
- Save tick 60 s, configurable.
Other 2
Notes
profileWorkersees both threads and the heap. No schema change, no SDK change.profileWorker(seconds, dir, thread):thread: 'main'profiles the CAP process's main thread (request handling, state-save pipeline, pollers) with the same in-thread profiler;'worker'stays the default. Both returnheapBefore/heapAfter(used/total/limit/external heap, malloced, RSS, ArrayBuffers in MB of that isolate) andgc(collections in the window, total ms, JSONbyKind). Shared implementationsrv/midnight/cpu-profile.ts. Live finding that asked for it: five minutes after a sponsored call the hosted worker sampled at GC 77 % / idle 14 % with the heap far below its 8 GB limit, and the main thread carried a steady 35 % of a core.profileWorkerfailures keep their message in production ($sanitize: false), a 503 no longer reads "Service Unavailable" only.
Other 5
Notes
Sponsor pool on one worker thread: the cheap fixes from the sharding analysis. No schema change, no SDK change.
- Prewarm serializes catch-up to the tip.
prewarmFeeSponsorPoolwaits for each sponsor to reach the chain tip (walletWaitForSyncedState) before starting the next one, so the first pool member is usable after 1/N of the wall clock instead of all N crawling together on the shared thread (measured: one wallet alone 113 events/s, three together ~26 each).NIGHTGATE_SPONSOR_PREWARM_SYNC_MScaps the wait per sponsor (default 30 min; 0 = build only, the old behaviour); a sponsor still behind is logged and the loop moves on. - Boot sweep keeps sessions holding a signing key.
closeSessionsFromPreviousProcesscloses viewing-only leftovers only; a session upgraded with a mnemonic (a pool member taken out of the config, a consumer's custodial wallet) stays active and is rebuilt lazily. Before, removing an id fromNIGHTGATE_FEE_SPONSOR_SESSIONand restarting revoked its key for good. - Idle progress watch. The worker peeks every facade's dust progress
every
NIGHTGATE_PROGRESS_WATCH_MS(default 60 s), refreshes the cachedgetWalletSyncProgresssnapshot and logsidle-sync ... behindEvents=while a facade is behind with no job waiting. A far-behind idle wallet no longer looks like a hang. getSponsorPoolStatus: per-sponsor status read cap 45 s (was 20 s;NIGHTGATE_SPONSOR_STATUS_TIMEOUT_MS). Three warm facades on one thread make a read take 5-30 s; 20 s reported healthy sponsors as not warm.- Admin
profileWorker(seconds?, dir?): CPU profile of the wallet worker thread with the in-thread V8 profiler (node:inspectorinside the worker, 1..120 s, worker keeps serving); returns self time by function/file, inclusive hot paths and the idle/GC/wasm shares (srv/midnight/cpu-profile-summary.ts), writes the raw.cpuprofilefor DevTools. Reference point: a warm facade at tip idles > 90 % (local measurement: 8 % of the worker for one facade, dust serialisation + GC).
- Prewarm serializes catch-up to the tip.
Other 2
Notes
CAP connection pool under load. No schema change, no SDK change.
features.use_generic_pooldefaults to true (src/cap-pool-default.ts, applied at plugin registration; an explicit host value stays). CAP's built-in pool (@cap-js/db-service3.0.x,generic-pool.js#dispense) loses a connection whenever it dispenses one to a request that already timed out: with the kind default of 1 s the pool emptied on api.nightgate.dev after a night of checkpoint stalls and every request failed withPool resource could not be acquired(2026-08-28 04:25 UTC).generic-pool(new dependency) does not have the defect; the repro is pinned against the installed db-service intest/unit/cap-db-pool.test.ts.- Standalone image, PostgreSQL:
requires.db.pool= max 20, acquire 30 s, destroy 5 s, idle 60 s, eviction 60 s, testOnBorrow;requires.db.clientconnectionTimeoutMillis10 s (docker/cds-config.mjs; CAP'scds_requires_db_pool_*env vars override). Composenightgate-postgres:shared_buffers=1GB max_wal_size=4GB checkpoint_timeout=15min wal_buffers=64MB.
Other 1
Notes
Sponsored deploys under a circuit floor. No schema change, no SDK change.
- Calls on a contract deployed under a grant (
deployedContracts) are exempt fromallowedCircuits: the effective policy carries the grant's deployed addresses asownContracts, the sponsor command passes them through and the worker's shape check skips the circuit list for a call on one of them. The contract list and the byte ceiling still apply. On a server withNIGHTGATE_SPONSOR_ALLOWED_CIRCUITS(or a policy file withallowedCircuits) the deploy used to land and every follow-up call was refused withcircuit '<name>' is not sponsorable(api.nightgate.dev,counter.increment, 2026-08-27).
- Calls on a contract deployed under a grant (
Other 5
Notes
PostgreSQL in the standalone image. No schema change, no SDK change.
NIGHTGATE_DB_URL.postgres://…selects PostgreSQL in the container (docker/cds-config.mjsbuilds the CDS config; unit-tested); the schema is deployed withcds deployon every boot (additive evolution),NIGHTGATE_DB_DEPLOY=neverskips it. Without the URL the SQLite path is unchanged. Compose gains apostgres:16-alpineunder--profile postgres(NIGHTGATE_PG_PASSWORD).nightgate-db-migrate(scripts/migrate-sqlite-to-postgres.mjs): copies every persisted entity of the loaded model from a SQLite file into a deployed, empty PostgreSQL database through CAP (0/1 -> boolean, JSON columns as text, views skipped;.textsand CAP's own tables such ascds_outbox_Messagesincluded), rows streamed via the statement iterator in batches of 500 (flat memory on large indexer databases), integers read as BigInt (Integer64exact; aDecimalSQLite already holds as a REAL beyond 2^53 aborts,scripts/migrate-values.mjs), source tables the model does not define are listed and abort with rows unless--ignore-unknown, per-table row-count check,--dry-run,--force; theSyncStatesingleton a previous boot wrote is replaced. Needs@cap-js/postgres+better-sqlite3in a plugin host (named in the error when missing). Documented invocation:docker compose run --rm --no-deps nightgate migrate --from /data/nightgate.db(same volume, network and env as the service).migratemode branches off before the server checks: it needs onlyNIGHTGATE_DB_URL(cds-config.mjs --db-only, noENCRYPTION_KEY, no HTTP password) and waits for the PostgreSQL listener (NIGHTGATE_DB_WAIT_SECONDS, default 60, validated 1..86400, per-attempt socket timeout capped at the time left) beforecds deploy.NIGHTGATE_DB_URL?sslmode=:disable,require(TLS unverified) orverify-full(chain + hostname,sslrootcert=<pem>);allow/prefer/verify-cacannot be honoured exactly by node-postgres and refuse, as do unknown modes.- Compose: no
:?required variables anymore.NIGHTGATE_PG_PASSWORDbroke the SQLite quickstart (compose interpolates before profile selection),ENCRYPTION_KEY/NIGHTGATE_HTTP_PASSWORDbrokeup proof-server; the containers refuse to start on an empty value instead (entrypoint,cds-config.mjs, postgres image). Imagemigratemode runs deploy + migration as a one-off container (docker run --rm ... <image> migrate --from /data/nightgate.db). - Compose default image tag 0.21.1.
Other 12
Notes
Runtime contract registration, per-grant sponsor policy, sponsored deploys, generation-pinned artifacts, honest 503/sync/dust reporting. Additive schema migration (six nullable
AgentGrantscolumns,ContractRegistrationstable). No vault redeploy, no circuit change.@odatano/nightgate-tx0.4.0.- Runtime contract registration. Admin
registerContract/unregisterContract/listContracts. Paths must lie insideNIGHTGATE_CONTRACTS_DIR(several roots, canonical, relative paths tried under each); module validated in a disposable worker thread, verifier keys andzkir/directory required; persisted inContractRegistrations, reloaded at boot, pinned by generation digest; register/unregister per name serialised. Config contracts are the immutable floor (409). - Sponsor policy per grant.
createAgentGrant(allowedContracts, allowedCircuits); effective policy of a token call = platform floor ∩ grant (absent grant list = floor, empty floor = grant, empty intersection =403 SPONSOR_POLICY_EMPTYat admission). Revoking the grant removes its reach. - Sponsor policy file.
NIGHTGATE_SPONSOR_POLICY_FILE: JSON{ allowedContracts, allowedCircuits, allowDeploy }, re-read per sponsored call behind an mtime cache, replaces the env lists while set. Invalid or missing file: last good policy, else503 SPONSOR_POLICY_UNAVAILABLE; a missing file is logged once. - Sponsored deploys. Grant
allowDeploy+maxDeploys(lifetime, default 1,deploysUsed); floorNIGHTGATE_SPONSOR_ALLOW_DEPLOYor policy fileallowDeploy. Shape check admits oneContractDeployper transaction underNIGHTGATE_SPONSOR_MAX_DEPLOY_BYTES(40960); maintenance updates refused. Budget reserved at the submit-intent in one DB transaction with thePendingSubmissionsrow (actionType: DEPLOY,deployed,deployReservationinsubmitIntentData) and the job transition (reportBroadcastOn). Rejected attempt: row closed, refund, hash cleared in one transaction (reportSubmissionRejectedOn); if that cannot commit, the job parks underREJECTED_ATTEMPT_BOOKKEEPING_PENDINGandsettleRejectedSponsorAttemptsre-runs the bookkeeping each reconciliation tick (job endsSPONSOR_ATTEMPT_REJECTED). Chain-failed deploys keep their reservation. Landed address recorded inAgentGrants.deployedContracts, added to the effective policy after the intersection; the reconciliation finalizer records it on both the crawler and the indexer-confirmer path. - txbuilder.
buildDeploySponsorable({ initialPrivateState, constructorArgs, witnesses, bind })returns the deploy transaction andcontractAddress(exactly one deploy action required).buildSponsorable({ calls, witnesses })batches on any contract under one shared witnesses object (per-callbeforehooks; the vault family may omit it).createTxBuilder({ zkConfigDir })uses localkeys/+zkir/(directories, verifier key per circuit, prover key + bzkir per circuit to prove); defaultcircuits= circuits ofcontractClass, else the vault's set.ZkAssetResult.source(remote|local),describeLocalZkAssets,readDeployAddress.@odatano/nightgate-tx0.4.0. - Artifact generations. Digest (
srv/submission/artifact-digest.ts) over module bytes,privateStateId, non-default slot width, prover and verifier keys, zkir, and module format (moduleFormat:commonjssection for CommonJS only; 0.20 digests of CommonJS artifacts accepted as legacy, ESM digests unchanged). Worker verifies the digest on disk, materialises an immutable snapshotNIGHTGATE_ARTIFACT_SNAPSHOT_DIR/<install>/<pid>/<digest>/ {module/artifact.mjs|.cjs,keys,zkir}(temp build, verify, rename;node_moduleslink for bare imports; foreignnode_modules= refuse) and imports class and assets from it only. Scaffold and zk/proving provider caches bounded byNIGHTGATE_WORKER_GENERATION_CACHE(8); snapshots released with their generation, dead-process roots and snapshots older thanNIGHTGATE_ARTIFACT_SNAPSHOT_TTL_DAYS(14) swept at first use; indexer WebSocket client shared per endpoint. Worker rotation afterNIGHTGATE_WORKER_MAX_GENERATIONS(32, 0 = never): admission closed (WORKER_ROTATING, retried by the client), in-flight calls complete, clean exit, respawn on next call;getWorkerStatus().rotationCount, metricwallet_worker_rotations. Main process imports no artifact for jobs (resolveContract(name, digest, { compile: true })only on request). - Retryable 503s.
JOB_ADMISSION_BUSY,WALLET_SYNCINGand retryable submission classifications keeperror.codeand message underNODE_ENV=production($sanitize: false) and setRetry-After, also onregisterForDustGeneration,deregisterFromDustGeneration,sendNight. - Sync progress.
getWalletSyncProgress:stale,staleSeconds(NIGHTGATE_SYNC_PROGRESS_STALE_S, 60),lastProgressAt, prewarmjobId/jobStatus,restoredFromSnapshot,snapshotSavedAt,facadeBuildStartedAt,facadeBuiltAt; INFO line RESTORED/COLD START undernightgate:facade. Facade origins dropped with the worker. - Prewarm bound. Fails on no
appliedIndexprogress forNIGHTGATE_PREWARM_STALL_MS(10 min, also when the wallet state cannot be read); absolute ceilingNIGHTGATE_PREWARM_SYNC_TIMEOUT_MS(12 h). - Dust race.
1010/170and1010/196are one retryable classification (srv/submission/dust-race.ts,transient: "dust-race"); bound deploy/call/batch paths rebuild-retry before a txHash exists (NIGHTGATE_DUST_RACE_RETRIES2,NIGHTGATE_DUST_RACE_BACKOFF_MS5 s). - Dust registration.
registerForDustGenerationreturnschanged,reason(already-registered|no-night-utxos, from the full coin set),requestedReceiver,dustReceiverAddressonly when applied,totalNightUtxos,registeredUtxosBefore,registeredUtxosAfter,settled,consolidated,message(NIGHTGATE_DUST_REGISTER_SETTLE_MS, 90 s). - Misc. Compose default image tag 0.21.0; snapshot copies the module's
source map with rebased
sourceRoot; worker clientRegisterDustGenerationOutcometype;sponsored-deploy:e2elane;burst-sponsor:e2etakesNIGHTGATE_AGENT_TOKEN.
- Runtime contract registration. Admin
Other 12
Notes
Monitoring surface. What a dashboard or a scraper needs was already known inside the process and unreachable from outside. Additive, with one migration.
-
Plain status routes.
getMetrics()builds Prometheus text and CAP wraps it as{"value": "# HELP ..."}, which no scraper parses; a container probe cannot express/api/v1/indexer/getReadiness()either. The same payloads are now served astext/plainand plain JSON from the same code (srv/monitoring/status.ts), and ready answers 200 or 503.They mount during CAP's bootstrap, BEFORE its authentication middlewares, so they are fail-closed: nothing mounts until an operator sets
NIGHTGATE_STATUS_TOKEN(bearer, constant-time compare) orNIGHTGATE_STATUS_ROUTES=public. And they are namespaced under/nightgate(NIGHTGATE_STATUS_ROUTES_PREFIX): CAP registers its own/healthright after that event, so a generic path would let a NIGHTGATE database problem decide an unrelated service's health. -
getRuntimeInfo(): version, network, proving mode, and two artifact digests per registered contract.artifactDigestis the generation this process loaded,currentDigestwhat is on disk now; a mismatch is exactly the state where every write job fails the generation guard until restart. Memoised behind a stat fingerprint (365 ms to 12.8 ms). -
getWorkerStatus(): wallet worker health, exit history included. The per-facade list is admin-only (it names the wallets this process holds); the count is not. Residency comes from the facade registry and is dropped when the worker exits, so a pool restored at the tip is neither under- nor over-reported. -
getSponsorPoolStatus(): fee-sponsor pool health in one call.dustNotesis the parallelism and counts FREE notes only. It is notregisteredNightUtxos, the sponsor's own NIGHT registered for dust generation: generation is delegable, so a foreign wallet pointing its NIGHT here grows the notes while the own count stays put (measured, 3 to 14). Never builds a cold facade, and not grantable to agent tokens. -
getJobStats()on the admin service, aggregating in SQL rather than over every row of the window. -
Readiness includes initialisation. A process whose
initialize()bailed answered ready:true whenever the crawler was disabled. It now requiresinitializedAND a non-offline mode, and a failed submission bootstrap counts as a failed initialisation rather than a warning. -
A worker crash and a planned stop are told apart. Both drop facade residency, since a facade lives in the worker. A crash then keeps the storage passphrases that state-saves the worker already delivered still need, because it cannot be waited on; a planned stop drains those saves and releases the passphrases, so a
shutdown()plus re-initialise in one process does not leave credentials of closed sessions referenced. -
One session-expiry rule (
srv/utils/session-expiry.ts), replacing nine bare date comparisons that each had to remember the platform-sponsor exemption. Facade eviction uses it too. -
Registered contracts with witnesses can be deployed. An unknown contract got an empty witness object, and a Compact constructor checks every declared name, so anything with a witness died before the deploy started. The fallback is now a stub that satisfies the check and throws when a circuit reaches for it.
-
Job admission survives a busy database. Five attempts over ~14 s instead of three over 5.5 s, and an exhausted budget answers 503 ("nothing was written, retry") instead of a bare 500. A workflow whose earlier step is already on chain escalates to
reconciliation_requiredrather than failing, and does so conservatively when the child state cannot be read either. -
Optional
labelon wallet sessions, settable atconnectWallet. Cosmetic, never used for lookup or authorisation. -
The image health-checks NIGHTGATE, not just its HTTP port (
docker/healthcheck.mjs); a 401 or an unexpected 404 is a failure.
Migration:
WalletSessionsgainslabel. Runnightgate-schema-deltabefore upgrading; the entrypoint only deploys a schema when the database file does not exist. A 0.19 database keeps the server offline with the missing column named, rather than failing on the firstconnectWallet.No SDK release: nothing under
packages/,src/txbuilder,src/browserorsrc/sdkchanged.-
Others providing
Compare all 7 on the OData page →- com.sap.cds:cds-adapter-odata-v2com.sap.cds:cds-adapter-odata-v2The OData V2 protocol adapter for CAP Java, which the runtime serves natively once this beta module is on the classpath, so a V2 frontend keeps working without a change to the CDS model.sap official, capOData V2 protocol adapter built into the CAP Java runtime
- com.sap.cds:cds-adapter-odata-v4com.sap.cds:cds-adapter-odata-v4The OData V4 protocol adapter for CAP Java, auto-exposing every Application Service in the model at /odata/v4/<Service> without protocol code of its own, active by default in the CDS Maven Archetype.sap official, capOData V4 protocol adapter for CAP Java Application Services
- @cap-js-community/odata-v2-adapter@cap-js-community/odata-v2-adapterExposes a CAP service as OData V2 for clients that cannot move to V4.community org, capExposes a CAP service as OData V2
- @odatano/core@odatano/coreA CAP plugin that serves the Cardano blockchain as OData V4, giving enterprise-shaped access to a public chain, funded through Cardano Catalyst.independent, capCAP plugin serving the Cardano blockchain as OData V4
- @root3287/cap-segw@root3287/cap-segwA CAP build tool that compiles a CDS service into the ABAP Model Provider and Data Provider classes a SAP Gateway SEGW project expects, for OData V4 by default or V2.independent, capCompiler from a CDS service to SEGW model and data provider classes
- @sap/cds@sap/cdsThe CAP Node.js runtime, which compiles the CDS model, serves the services declared in it, answers generic requests before any handler runs, and loads every cds-plugin.sap official, capCAP Node.js runtime
Details
Sources
Sources are graph entities: the same page may back several entries, and an unreachable source flags every row that rests on it.
Compatibility and provenance
Flavors
“npm install @odatano/nightgate @cap-js/sqlite” — ODATANO/NIGHTGATE — README
Contributors
History in the atlas
- Indexes the Midnight blockchain and submits transactions to it from CAP, exposed as OData V4, with command-line tools for schema deltas, key rotation and migration.
- First seen.
For your agent
- Ref
artifact:odatano-nightgate- JSON
- /artifacts/odatano-nightgate.json
- MCP
- Connecting an agent