Beta's cache-first work added connection-test.service.spec.ts, whose import
graph (connection/client.config.ts) reads process.env.INFACTORY_URL at load
time. The Dockerfile test target only set DUC_URL, so that suite crashed at
import ("Cannot read properties of undefined (reading 'startsWith')") in CI and
in any bare `npm test` run.
Add ENV INFACTORY_URL=0.0.0.0:50052 alongside the existing DUC_URL, matching the
local-connection convention (0.0.0.0 => no SSL). Full suite: 52 passed, 5 skipped.
Co-Authored-By: WOZCODE <contact@withwoz.com>
- platform-api.controller.spec: addCdcTable expectations now include the CDC
fields the controller threads (iceberg_table_name, iceberg_qualify_table_name,
column_exclude_list) which were added by the CDC-iceberg work.
- release_note specs: provide DadosferaLogger mock — ReleaseNoteService gained an
@Inject(DadosferaLogger) dependency (from beta) without its specs being updated,
so they failed DI resolution on merge.
Full suite: 52 passed, 5 skipped, 0 failed.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Beta's refresh-catalog RefreshCatalogReq DTO restricted plugin to
oracle/mysql/postgresql/sqlserver. Under the cache-first catalog model
(adopted for CDC in the beta merge), the CDC schema-fetch flow posts
/connection-test/refresh-catalog with plugin=mysql_cdc, which the @IsIn
rejected ("plugin must be one of: oracle, mysql, postgresql, sqlserver").
Add mysql_cdc/postgresql_cdc/oracle_cdc to the @IsIn and @ApiProperty enum,
matching the platform connection-test SQS plugin set. docsfera.json regenerated.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Resolves conflicts for PR #510 (base: beta):
- package.json: keep protospack ^3.40.0-beta.20 (carries CDC→Iceberg fields);
package-lock.json reconciled (protospack was the only dep delta vs beta).
- connection-test controller/service/dto: keep BOTH feature sets — our CDC
prerequisites validation AND beta's refresh-catalog endpoints.
- Adopt beta's cache-first catalog reads (connections-api proxy) over our gRPC
path. connectionTestListTables now enriches each table with primary_keys
derived from the cached columns endpoint (is_primary_key), preserving the
CDC create flow's need for PKs under the new architecture; spec updated.
- docsfera.json resolved to ours; regenerated on next app bootstrap.
connection-test spec: 6 passed. tsc --noEmit: clean.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Replaces the local file:../protospack-v2/...cdc-iceberg.4.tgz tarball reference
with the published CodeArtifact version ^3.40.0-beta.20 (carries the CDC→Iceberg
qualify_namespace / iceberg_qualify_table_name / column_exclude_list proto fields).
Co-Authored-By: WOZCODE <contact@withwoz.com>
PR #513 merged to beta but no semantic-release ran: its commits used
conventional-commits prefixes (feat(auth):, fix(auth):) which the
.releaserc.json eslint preset does not recognise, so commit-analyzer
found no release-worthy change. This empty FIX: commit matches the
eslint preset's releaseRules (tag FIX -> patch) to cut a beta release
that includes the /auth/me permission-seqids change, so stg can deploy it.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Return payload.permissions verbatim (numeric seqids) instead of
translating them to claim strings. Consumers own the seqid->meaning
mapping. Drops permission-claims.ts entirely; UserDTO.permissions is
now number[].
Co-Authored-By: WOZCODE <contact@withwoz.com>
Keep Maestro a pure identity provider: /auth/me exposes the user's
permission claim strings and nothing consumer-specific. Consumers derive
whatever meaning they need (roles, module access, groups) from the claim
vocabulary — claims are already namespaced group:action.
- UserDTO: drop roles[]/modules[], keep permissions[].
- Helper shrinks to a generic seqid->claim translation
(orchest-identity.ts -> permission-claims.ts, translateSeqidsToClaims).
- api-key branch: permissions: [] only.
The roles/modules derivation moves entirely to the consumer (Orchest's
auth-server adapter).
Co-Authored-By: WOZCODE <contact@withwoz.com>
The JWT `permissions` claim is an array of numeric seqids at runtime
(see authentication.guard.ts / authentication.decorator.ts), not claim
strings. deriveOrchestIdentity previously matched claim strings against
this numeric array, so roles[]/modules[] were always empty for every
real user.
- deriveOrchestIdentity now takes number[] | undefined and matches
seqids sourced from PERMISSIONS_GROUPS (permissions.enum.ts) instead
of hand-copied literals.
- permissions is translated back to claim strings via a full
seqid->claim catalog built once from PERMISSIONS_GROUPS; unknown
seqids are dropped (auth-server ignores permissions[] in v1).
- auth.controller.ts's api-key branch literal is now annotated
`: UserDTO` so tsc enforces the three fields there.
- Both spec files re-fixtured with numeric seqid inputs, including a
mixed admin+module case and an exact claim-string translation
assertion.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Threads the new CDC→Iceberg fields from the REST DTOs to the gRPC calls:
- CdcTableReq.iceberg_qualify_table_name + IcebergDestinationReq.qualify_namespace
in input.model.ts
- inputs.service.ts create map forwards iceberg_qualify_table_name
- platform-api.controller.ts addTable body + cdcTable thread iceberg_qualify_table_name
Bumps protospack to v3.41.0-cdc-iceberg.4; docsfera.json regenerated with the new
/platform/iceberg/{namespaces,tables/validate} routes.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Adds CdcColumnReq {name, type, is_primary_key} and columns? on
CdcTableReq, forwarded through createCdc and the addTable (Edit
Objects add-table) path so the column schema reaches in-factory for
Iceberg deduped-table pre-create. Bumps protospack-v2 to
3.41.0-cdc-iceberg.1, which adds the matching CdcColumn field
(now required on CdcTable) and regenerates docsfera.json.
Co-Authored-By: WOZCODE <contact@withwoz.com>
InputsService.createCdc builds InputCreateCdcRequest by enumerating
fields (not spreading), so a revert of the destination/iceberg_table_name
mapping lines would not be caught by the existing controller spec, which
only mocks InputsService. Add a unit test at the service boundary that
asserts destination and per-table iceberg_table_name reach the gRPC
request, plus a back-compat case with no destination.
Also mark CdcTableReq.iceberg_table_name as advisory/reserved: the
platform derives the Iceberg table name itself today and does not yet
consume this field.
Co-Authored-By: WOZCODE <contact@withwoz.com>
The POST /pipelines/:pipelineId/inputs/:inputId/tables route built its
CdcTable payload field-by-field and silently dropped iceberg_table_name
even though inputsService.addCdcTable/the gRPC AddCdcTable call (and the
protospack CdcTable message) already support it. Widen the inline request
body type and thread the field into the addCdcTable payload; absent for
snowflake, unchanged back-compat.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Unblocks the CdcDestination contract locally on the CDC branch. Swap to a
published registry version before merge (CI guard blocks file: deps).
Co-Authored-By: WOZCODE <contact@withwoz.com>
Removing N tables via Edit Objects previously looped a single-table
DELETE per table (maestro deleteTable hardcodes job_ids:[one]), so the
Debezium source + Snowflake sink connectors were rewritten/restarted once
per table. platform-api's DELETE /pipeline/:id/jobs already batches (2
connector writes total for any N), but nothing above it used the array.
New maestro DELETE /pipelines/:pipelineId/inputs/:inputId/tables takes
{ table_names: [] }: soft-deletes each in DynamoDB (tracking successes),
resolves all table_names -> job_ids from the platform pipeline in one GET,
then makes ONE DELETE /pipeline/:id/jobs with all job_ids. All-or-nothing:
any failure (a later mark, an unmatched table, or the platform delete)
rolls back only the marks made in this call.
The single-table deleteTable route is kept (unchanged) — nothing else
depends on removing it, and that's a separable cleanup.
Tests: N tables -> one platform DELETE with all job_ids and no per-job
call; rollback on platform failure; rollback + no delete when a later mark
fails; 404 for an unmatched table. 8 controller specs pass; maestro builds.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Switch @dadosfera/protospack-v2 from the local file: tarball (removed from
the protospack-v2 repo) to the CI-published CodeArtifact version. Installs
and builds clean; test suites pass.
Co-Authored-By: WOZCODE <contact@withwoz.com>
The guard read `status[status.length-1].last_status` on the pipeline_run
history. For a pipeline that has never run — which is the permanent state
of CDC pipelines, since they replicate continuously and never record
batch runs — pipeline_run is empty, so the last element is undefined and
`.last_status` threw, surfacing as "Error checking pipeline status:
Cannot read properties of undefined (reading 'last_status')". This
blocked Edit Objects (and delete-table) for every CDC pipeline.
Treat an empty/statusless run history as "not running" and allow the
edit (undefined never satisfied 'running' anyway). Also stop the catch
from double-wrapping the deliberate is-running BadRequestException, so
that rejection keeps its clear message; genuine status-check failures
still fail closed with the wrapped message (safe default for a
destructive-op gate).
Adds a guard spec: empty history -> allow, statusless -> allow, not
running -> allow, running -> is-running message (not wrapped), platform
error -> wrapped message.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Claude-Session: https://claude.ai/code/session_01145m1zZMfx8RSJBxAhySdg
The pipeline create body carries `config.tables[].destinations`, which
the CDC path in pi-factory needs to honor a user-supplied raw Snowflake
table name. The gRPC PipelineV2CreateRequest previously had no `config`
field, so `...body` dropped it on the wire.
Bump protospack to cdc.3 (adds optional `config` string). Serialize
`body.config` into the create request the same way `properties` is
handled, and add `config?` to the ICreatePipelineV2Req DTO so it's
typed. Add a spec asserting the gRPC request carries a stringified
config with the destination intact.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Claude-Session: https://claude.ai/code/session_01145m1zZMfx8RSJBxAhySdg
updatePlatformJobs pushes batch sync_mode/memory to jobs by positional
index — meaningless for CDC and corrupting. CDC add/remove use dedicated
endpoints, so skip updatePlatformJobs for CDC inputs. The input record
update still runs.
Co-Authored-By: WOZCODE <contact@withwoz.com>
deleteTable called the DB-row-only DELETE /jobs/{id}, leaving the CDC
connector still replicating a removed table. For CDC jobs, call
DELETE /pipeline/{id}/jobs (RemoveJobsUsecase) with delete_snowflake_tables
false so replication stops but landed data is kept. Batch path unchanged.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Rafael Santana
marcos.rodrigues