ICPI Env Cutover Rehearsal

Goal

Rehearse the first real env-only gateway cutover from old colocated ICPI hosting to new svc-icpi, while keeping:

• gateway route shape unchanged
• auth behavior unchanged
• rollback as an env flip

Runtime Layout Used

• svc-auth: http://localhost:4100
• old colocated ICPI host: http://localhost:4020
• new extracted ICPI host: http://localhost:4021
• gateway: http://localhost:4001

Env Settings Used

Old-host direct runtime

• PORT=4020
• DATABASE_URL=postgres://kvary:kvary@127.0.0.1:5432/kvary
• JWT_SECRET=dev-secret-change-me
• AUTH_SERVICE_URL=http://127.0.0.1:4100

New extracted runtime

• PORT=4021
• DATABASE_URL=postgres://kvary:kvary@127.0.0.1:5432/kvary
• JWT_SECRET=dev-secret-change-me
• AUTH_SERVICE_URL=http://127.0.0.1:4100

Gateway fallback mode

• PORT=4001
• AUTH_SERVICE_URL=http://127.0.0.1:4100
• RISK_SERVICE_URL=http://127.0.0.1:4200
• RISK_INTERNAL_SVC_KEY=dev-internal
• JWT_SECRET=dev-secret-change-me
• TENDERS_SERVICE_URL=http://127.0.0.1:4020
• BUTKHUZI_SERVICE_URL=http://127.0.0.1:4020
• ICPI_SERVICE_URL unset

This exercises the current fallback seam:

• ICPI_SERVICE_URL ?? TENDERS_SERVICE_URL

Gateway cutover mode

• same as fallback mode, except:
• ICPI_SERVICE_URL=http://127.0.0.1:4021

Cutover Steps Rehearsed

1. Keep both ICPI runtimes up:
   • old svc-tenders ICPI host on :4020
   • new svc-icpi on :4021
2. Start gateway in fallback mode without ICPI_SERVICE_URL.
3. Verify GET /api/v1/icpi/* and protected POST /api/v1/icpi/upsert against old-host parity.
4. Stop gateway only.
5. Restart gateway with ICPI_SERVICE_URL=http://127.0.0.1:4021.
6. Re-run the same gateway route checks.
7. Compare gateway responses to direct svc-icpi responses.

Rollback Steps Rehearsed

1. Stop gateway.
2. Remove the ICPI_SERVICE_URL override.
3. Keep TENDERS_SERVICE_URL=http://127.0.0.1:4020.
4. Restart gateway.
5. Recheck one read route and one protected write denial path against the old colocated host.

Observations

• The gateway route surface stayed unchanged across fallback, cutover, and rollback.
• The gateway auth surface stayed unchanged across fallback, cutover, and rollback.
• Env-only cutover was sufficient to redirect the gateway to svc-icpi.
• Rollback required only a gateway restart with the prior env shape.
• Old colocated hosting remained intact the entire time.

Verified Behaviors During Rehearsal

Fallback mode

• gateway read routes matched old host exactly
• gateway POST /api/v1/icpi/upsert missing-token denial matched old host exactly
• gateway POST /api/v1/icpi/upsert unresolved-principal denial matched old host exactly

Cutover mode

• gateway read routes matched new svc-icpi exactly
• gateway POST /api/v1/icpi/upsert missing-token denial matched new svc-icpi exactly
• gateway POST /api/v1/icpi/upsert unresolved-principal denial matched new svc-icpi exactly

Rollback mode

• gateway read route matched old host again after env rollback
• gateway protected write denial matched old host again after env rollback

Remaining Risks

• Fully authenticated happy-path upsert parity was not exercised in this sprint because no seeded stable principal/session fixture was introduced.
• svc-auth showed a separate robustness issue when /auth/me received a valid-signature token whose sub was not UUID-shaped.
• The gateway still depends on correct env discipline at process start; cutover is low risk, but not hot-reload automatic.

Rollback Verdict

Rollback remains simple and low-risk:

• stop gateway
• remove or change ICPI_SERVICE_URL
• restart gateway

No ICPI code move-back is required for rollback at this stage.