Commit Graph
1006 Commits
Author SHA1 Message Date
RafaelandWOZCODE f1d539a912 fix: PipelineExecutionGuard allows edits when pipeline has no run history
The guard read `status[status.length-1].last_status` on the pipeline_run
history. For a pipeline that has never run — which is the permanent state
of CDC pipelines, since they replicate continuously and never record
batch runs — pipeline_run is empty, so the last element is undefined and
`.last_status` threw, surfacing as "Error checking pipeline status:
Cannot read properties of undefined (reading 'last_status')". This
blocked Edit Objects (and delete-table) for every CDC pipeline.

Treat an empty/statusless run history as "not running" and allow the
edit (undefined never satisfied 'running' anyway). Also stop the catch
from double-wrapping the deliberate is-running BadRequestException, so
that rejection keeps its clear message; genuine status-check failures
still fail closed with the wrapped message (safe default for a
destructive-op gate).

Adds a guard spec: empty history -> allow, statusless -> allow, not
running -> allow, running -> is-running message (not wrapped), platform
error -> wrapped message.

Co-Authored-By: WOZCODE <contact@withwoz.com>
Claude-Session: https://claude.ai/code/session_01145m1zZMfx8RSJBxAhySdg
2026-08-17 13:35:19 -03:00
RafaelandWOZCODE c85e2ac5da feat: forward create body config over gRPC (cdc.3) for CDC destinations
The pipeline create body carries `config.tables[].destinations`, which
the CDC path in pi-factory needs to honor a user-supplied raw Snowflake
table name. The gRPC PipelineV2CreateRequest previously had no `config`
field, so `...body` dropped it on the wire.

Bump protospack to cdc.3 (adds optional `config` string). Serialize
`body.config` into the create request the same way `properties` is
handled, and add `config?` to the ICreatePipelineV2Req DTO so it's
typed. Add a spec asserting the gRPC request carries a stringified
config with the destination intact.

Co-Authored-By: WOZCODE <contact@withwoz.com>
Claude-Session: https://claude.ai/code/session_01145m1zZMfx8RSJBxAhySdg
2026-08-17 11:33:29 -03:00
RafaelandWOZCODE 0d771d1e4a FIX: skip batch job-updates for CDC in updatePipelineInput
updatePlatformJobs pushes batch sync_mode/memory to jobs by positional
index — meaningless for CDC and corrupting. CDC add/remove use dedicated
endpoints, so skip updatePlatformJobs for CDC inputs. The input record
update still runs.

Co-Authored-By: WOZCODE <contact@withwoz.com>
2026-08-16 22:04:00 -03:00
RafaelandWOZCODE 7bd269950f FIX: CDC table removal reconfigures the Debezium connector
deleteTable called the DB-row-only DELETE /jobs/{id}, leaving the CDC
connector still replicating a removed table. For CDC jobs, call
DELETE /pipeline/{id}/jobs (RemoveJobsUsecase) with delete_snowflake_tables
false so replication stops but landed data is kept. Batch path unchanged.

Co-Authored-By: WOZCODE <contact@withwoz.com>
2026-08-16 22:00:39 -03:00
RafaelandWOZCODE 017fd145a9 CHORE: bump protospack-v2 to cdc.2 + pass name through createCdc
CdcTable.name is now required; map it (== table_name) in maestro createCdc.

Co-Authored-By: WOZCODE <contact@withwoz.com>
2026-08-16 21:56:34 -03:00
RafaelandWOZCODE 27e0dedea4 feat(maestro): expose tables[].primary_keys on /connection-test/tables
Co-Authored-By: WOZCODE <contact@withwoz.com>
2026-08-16 16:57:29 -03:00
RafaelandWOZCODE a597c41070 feat(cdc): expose CDC routes over REST (on main + beta protospack)
Re-applied onto fresh origin/main. pipelinesV2: live-status/pause/unpause/
restart/reset-state routes (per-route @RequireSomePermission — GET for status,
UPDATE for mutations — matching main's current auth convention). connection-test:
POST /connection-test/cdc-prerequisites. inputs: POST /inputs/cdc -> InputCreateCdc.
Consumes protospack 3.40.0-beta.15-cdc.0 tarball; CI guard added.

Co-Authored-By: WOZCODE <contact@withwoz.com>
2026-08-15 19:45:09 -03:00
marcos.rodrigues 0c4888ccdb FIX: require collect module in endpoints 2026-08-12 15:02:37 -03:00
marcos.rodrigues b89909ad66 FIX: skip nimbus update when customer haven't catalog module 2026-08-07 14:48:46 -03:00
viniciusgadea 3fd586753e FEAT: add endpoint and logic to update data asset certification status 2026-07-27 17:07:19 -03:00
viniciusgadea 4a5f8f679a FEAT: rename documentation_status to certification_status in docs and update package.json for protospack versioning 2026-07-27 17:06:58 -03:00
viniciusgadea a444f5e5ec FEAT: add documentation status enum and property to data asset 2026-07-27 17:06:25 -03:00
marcos.rodrigues 0d30c1cf83 FEAT: remove deprecated protospack lib 2026-07-27 17:06:10 -03:00
marcos-silva-rodrigues db2c7d6c02 FIX: tests 2026-05-25 17:43:38 -03:00
marcos-silva-rodrigues 39c66f030a FIX: release note endpoint 2026-05-25 14:51:32 -03:00
marcos-silva-rodrigues 1223e21ac4 FEAT: pipeline upgrade route 2026-05-25 14:51:20 -03:00
marcos-silva-rodrigues e32787baff FIX: roles 2026-04-29 17:06:04 -03:00
marcos-silva-rodrigues 02627d023a FEAT: list assets by pipeline and multiple ids 2026-04-29 16:26:43 -03:00
marcos-silva-rodrigues e980c58507 FEAT: list schema and filter assets by schema 2026-04-23 15:15:14 -03:00
marcos-silva-rodrigues 0cb9da13d0 FIX: block cancel first pipeline 2026-04-16 14:58:19 -03:00
marcos-silva-rodrigues 0d627b0451 FEAT: guard to prevent pipeline update when pipeline is running 2026-04-13 14:23:44 -03:00
marcos-silva-rodrigues 901518d26c FIX: remove full table 2026-04-09 17:38:28 -03:00
marcos-silva-rodrigues 4ab8147149 FEAT: remove endpoints 2026-04-09 17:23:58 -03:00
marcos-silva-rodrigues 0086528d51 FEAT: update connectors 2026-04-07 18:05:30 -03:00
viniciusgadea efb0b58648 FEAT: update deleteTable endpoint to use pipelineId in path and refactor parameters 2026-04-02 17:28:42 -03:00
viniciusgadea 8a9b58b612 FIX: rename method normalizeJobId to normalizePipelineId for clarity 2026-04-02 16:40:35 -03:00
viniciusgadea 837a9d7265 FEAT: update job deletion logic to mark tables as deleted and adjust API endpoints accordingly 2026-04-02 15:40:07 -03:00
viniciusgadea d0122a9c20 FEAT: remove unused status field from InputDocument and updateInputTable method 2026-04-01 14:43:51 -03:00
viniciusgadea 968f75b688 FEAT: update job deletion endpoint to include inputId in path and implement rollback for table deletion 2026-04-01 14:26:55 -03:00
viniciusgadea 877cb9d281 FEAT: refine PipelineTablesConfig type definition and update parsing logic in PipelinesController 2026-03-31 18:38:25 -03:00
viniciusgadea 59efdb6272 FEAT: update job deletion and mark associated table as deleted in DynamoDB. Update protospack-v2 2026-03-31 16:36:59 -03:00
viniciusgadea ecca9106f0 FEAT: remove PipelinesV2Module from PlatformApiModule imports 2026-03-31 11:17:32 -03:00
viniciusgadea efc1f49d92 FEAT: update delete job endpoint summary and remove unused InputsModule from platform-api module 2026-03-31 10:55:31 -03:00
viniciusgadea 2d46ac3213 FEAT: add delete job endpoint and update related services; update protospack-v2 version to 3.40.0-beta.3 2026-03-31 10:54:50 -03:00
marcos-silva-rodrigues dcf7aed51c MERGE: resolve package conflicts 2026-03-30 12:05:36 -03:00
marcos-silva-rodrigues 7fbce8b5be FEAT: update input 2026-03-30 12:02:27 -03:00
viniciusgadea b640465624 Merge remote-tracking branch 'origin/beta' into feat/cancel-pipeline-run-beta 2026-03-26 11:19:30 -03:00
viniciusgadea 98282471a9 FIX: remove unused data asset methods from ElasticsearchService 2026-03-26 11:08:40 -03:00
viniciusgadea 8983430889 REF: remove last_run_status tracking and related Elasticsearch update logic from pipeline cancellation 2026-03-26 10:32:09 -03:00
viniciusgadea d3aeca12cd FIX: remove last_run_canceled_at from PipelineDocument and update last_run_status handling 2026-03-26 10:32:09 -03:00
viniciusgadea ce619942a3 FEAT: add endpoint to cancel running pipeline runs and update Elasticsearch status 2026-03-26 10:32:09 -03:00
viniciusgadea bf4f3cfd8c FEAT: update CustomerSidebarSection and related DTOs to use object type for title; update protospack-v2 version; update from id to customerId 2026-03-25 15:40:57 -03:00
viniciusgadeaandClaude Sonnet 4.6 01afa69fcb FIX: remove duplicate identifier_columns declaration in TableColumns DTO
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 15:37:15 -03:00
viniciusgadea 80f3913fd2 FEAT: update CustomerSidebarSection to support both menu and link items 2026-03-25 15:24:59 -03:00
viniciusgadea 10293ad6a9 FEAT: update customer links handling and DTOs for improved structure sidebar. update protospack-v2 2026-03-25 15:24:59 -03:00
marcos-silva-rodrigues 21c57e5620 FEAT: update nimbus after input 2026-03-24 12:28:20 -03:00
Marcos Rodrigues Silva 13903b9bb9 Merge pull request #463 from dadosfera/fix/user-from-me-endpoint
FIX: send correct name and email from user payload
2026-03-19 14:36:58 -03:00
marcos-silva-rodrigues 2e3a13d421 FIX: send correct name and email from user payload 2026-03-19 14:35:14 -03:00
vinicius gadea 254a638392 Merge pull request #457 from dadosfera/feat/sidebar-menu
Feat/sidebar menu
2026-03-19 10:41:12 -03:00
viniciusgadea 74b3bd6b46 FEAT: update CustomerSidebarSection and related DTOs to use object type for title; update protospack-v2 version; update from id to customerId 2026-03-19 08:14:56 -03:00