PR #513 merged to beta but no semantic-release ran: its commits used
conventional-commits prefixes (feat(auth):, fix(auth):) which the
.releaserc.json eslint preset does not recognise, so commit-analyzer
found no release-worthy change. This empty FIX: commit matches the
eslint preset's releaseRules (tag FIX -> patch) to cut a beta release
that includes the /auth/me permission-seqids change, so stg can deploy it.
Co-Authored-By: WOZCODE <contact@withwoz.com>
Return payload.permissions verbatim (numeric seqids) instead of
translating them to claim strings. Consumers own the seqid->meaning
mapping. Drops permission-claims.ts entirely; UserDTO.permissions is
now number[].
Co-Authored-By: WOZCODE <contact@withwoz.com>
Keep Maestro a pure identity provider: /auth/me exposes the user's
permission claim strings and nothing consumer-specific. Consumers derive
whatever meaning they need (roles, module access, groups) from the claim
vocabulary — claims are already namespaced group:action.
- UserDTO: drop roles[]/modules[], keep permissions[].
- Helper shrinks to a generic seqid->claim translation
(orchest-identity.ts -> permission-claims.ts, translateSeqidsToClaims).
- api-key branch: permissions: [] only.
The roles/modules derivation moves entirely to the consumer (Orchest's
auth-server adapter).
Co-Authored-By: WOZCODE <contact@withwoz.com>
The JWT `permissions` claim is an array of numeric seqids at runtime
(see authentication.guard.ts / authentication.decorator.ts), not claim
strings. deriveOrchestIdentity previously matched claim strings against
this numeric array, so roles[]/modules[] were always empty for every
real user.
- deriveOrchestIdentity now takes number[] | undefined and matches
seqids sourced from PERMISSIONS_GROUPS (permissions.enum.ts) instead
of hand-copied literals.
- permissions is translated back to claim strings via a full
seqid->claim catalog built once from PERMISSIONS_GROUPS; unknown
seqids are dropped (auth-server ignores permissions[] in v1).
- auth.controller.ts's api-key branch literal is now annotated
`: UserDTO` so tsc enforces the three fields there.
- Both spec files re-fixtured with numeric seqid inputs, including a
mixed admin+module case and an exact claim-string translation
assertion.
Co-Authored-By: WOZCODE <contact@withwoz.com>
- Fix proxy path: /jobs/jdbc/:jobId/rename-tables → /jobs/:jobId/rename-tables
- Remove pipeline_id from ES data asset lookup, search by table_name + table_schema only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add POST /platform/jobs/:jobId/rename-tables that renames Snowflake
tables via platform-api and syncs the rename to Elasticsearch and
Nimbus (table-metadata, column-metadata, data-preview). If catalog
sync fails, all completed catalog steps are rolled back in reverse
order and the Snowflake rename is reverted.
- Support any connector type (jdbc, singer, s3) via getJobByAnyConnectorType
- Resolve old table names from output_config (raw/qualify)
- Skip qualify sync when output_config.qualify has no table_name
- Add findDataAssetByPipelineAndTable and updateDataAsset to ElasticsearchService
- Add renameTableOnNimbus, renameColumnMetadataOnNimbus, renameDataPreviewOnNimbus to CatalogService
- Add upstream error logging to PlatformApiService
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This fix addresses a breaking change introduced in class-validator 0.14.0
where the default for forbidUnknownValues changed from false to true.
Issue:
- POST /connections was returning 400 "an unknown value was passed to the
validate function" errors
- This occurred because CreateConnectionDto and UpdateConnectionDto have no
validation decorators, causing class-validator 0.14.0+ to treat them as
"unknown values"
- Extra fields (like connector_version) in request payloads would fail
validation
Solution:
- Set forbidUnknownValues: false to allow DTOs without validation decorators
- Set whitelist: true to automatically strip extra properties not defined
in DTOs
- This maintains backward compatibility while adding security by removing
unexpected fields
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This fix addresses a breaking change introduced in class-validator 0.14.0
where the default for forbidUnknownValues changed from false to true.
Issue:
- POST /connections was returning 400 "an unknown value was passed to the
validate function" errors
- This occurred because CreateConnectionDto and UpdateConnectionDto have no
validation decorators, causing class-validator 0.14.0+ to treat them as
"unknown values"
- Extra fields (like connector_version) in request payloads would fail
validation
Solution:
- Set forbidUnknownValues: false to allow DTOs without validation decorators
- Set whitelist: true to automatically strip extra properties not defined
in DTOs
- This maintains backward compatibility while adding security by removing
unexpected fields
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add auth_provider to values.yaml (cognito/keycloak)
- Add AUTH_PROVIDER env var to deployment template
- Note: maestro only needs to know which provider is used, duc handles connection
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add auth_provider config to values.yaml (cognito/keycloak)
- Add AUTH_PROVIDER env var to deployment template
- Add conditional Keycloak env vars (URL, realm, client ID, secret)
- Enables parallel deployments with different auth providers
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add authProvider to user entity DTO
- Update user service to include authProvider
- Update auth controller response
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update multer override from 1.4.5-lts.1 to 2.0.2 to fix HIGH vulnerabilities:
- CVE-2025-47935 (DoS via memory leaks)
- CVE-2025-47944
- CVE-2025-48997
- CVE-2025-7338
Reduces HIGH vulnerabilities from 4 to 0
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Pass TOTP code to DUC for Keycloak users with MFA
- Add Dockerfile.local for local protospack builds
- Add .dockerignore to exclude node_modules from Docker context
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Users no longer need to provide customer_id in the request body for
execute, pause, and unpause pipeline operations - it's now automatically
injected from the authenticated user's session.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The protobuf definition expects reference_column to be an object with
name and type fields, but it was being stored as just a string (column
name). This caused pipeline fetching to fail with the error:
".NewTable.reference_column: object expected"
Changes:
- Update ReferenceColumn interface in DynamoDB service
- Update extractTablesFromJobs to create reference_column object
- Update syncJobInputToDynamoDB to handle reference_column object
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The protobuf definition expects reference_column to be an object with
name and type fields, but it was being stored as just a string (column
name). This caused pipeline fetching to fail with the error:
".NewTable.reference_column: object expected"
Changes:
- Update ReferenceColumn interface in DynamoDB service
- Update extractTablesFromJobs to create reference_column object
- Update syncJobInputToDynamoDB to handle reference_column object
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The type field was missing from the createPipeline call,
causing ES documents to not have the type field set.
Maps: jdbc->database, singer->application, s3->file
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The type field was missing from the createPipeline call,
causing ES documents to not have the type field set.
Maps: jdbc->database, singer->application, s3->file
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add normalization for pipelineId and runId in getPipelineRuns, getPipelineRun, and getPipelineRunLogs
- Remove unused createPipelineRun route
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add ELASTICSEARCH_URL and ELASTICSEARCH_API_KEY env vars
- Pull credentials from {env}/microservices/elasticsearch secret
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add schedule limit validation against customer's scheduleLimit from DUC
- Improve ES update to only update provided fields
- Mark GET /platform/pipeline/:pipelineId/pipeline_run as READY
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add cron-validation utility with Airflow presets support (@once, @daily, etc.)
- Validate cron expressions against customer scheduleLimit from DUC
- Fetch scheduleLimit via gRPC to ensure up-to-date configuration
- Improve ES updatePipeline to fetch current doc and merge changes
- Only update fields explicitly provided in PATCH request body
- Preserve config.tables when updating config.cron
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Platform API module to proxy requests to Platform API service
- Add Elasticsearch service for pipeline catalog sync (CRUD operations)
- Add DynamoDB service for storing pipeline tables/inputs configuration
- Sync pipeline creation/update/delete to Elasticsearch index
- Extract tables from jobs with connector-specific mappings:
- JDBC: table_name, load_type, column_include_list, incremental_column_name
- Singer: replication_method -> full_load/incremental
- S3: same as Singer
- Map connector types to DynamoDB types (jdbc->database, singer->application, s3->file)
- Validate connector type is provided in job input
- Normalize pipeline IDs for Platform API (replace - with _), keep UUIDs for ES
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Change TLS check from ENV !== 'local' to REDIS_TLS === 'true'
- Add REDIS_TLS env var to Helm deployment template
- Default to true for OCI Redis, can override for self-hosted K8s
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Resolve ingress.yaml timeout annotation conflicts by keeping both configurations
- Resolve stg.yaml conflicts by merging affinity/resources config with restricted_ip
- Fix chart references in unimed-maestro and private-maestro sections
- Update all chart paths from ../maestro to ../helm-chart for consistency
This merge brings in the latest beta changes while preserving our Oracle migration
and affinity standardization configuration.
- Change trigger branches from main/stg to main/beta
- Update environment extraction to map beta → stg environment
- Align with standard branch naming convention (beta for staging)
- Consistent with deploy-manually.yml workflow configuration
Oracle Migration Changes:
- Create deploy/helm-chart/ directory structure (moved from maestro/)
- Create deploy/helmfiles/ directory structure (moved from helmfiles/)
- Update chart references in helmfiles from ../maestro to ../helm-chart
- Update deploy-manually.yml from Azure AKS to Oracle OKE
- Update validate-k8s.yml from Azure AKS to Oracle OKE
- Replace Azure CLI with OCI CLI installation and configuration
- Replace Azure authentication with OCI authentication using secrets
- Replace az aks get-credentials with oci ce cluster create-kubeconfig
- Update helmfile paths from helmfiles/ to deploy/helmfiles/
- Remove DockerHub integration (push_to_dockerhub input and related steps)
- Change runner from [self-hosted, prd-azure] to [self-hosted, prd-oracle]
- Add environment field for proper Oracle deployment environment handling
- Add HOME environment variable for OCI CLI
Affinity/Resources Standardization:
- Add affinity configuration with name=general node selector in values.yaml
- Update deployment template to use conditional affinity and resources blocks
- Configure staging to disable both affinity and resources (null values)
- Enable production to use affinity targeting name=general nodes
- Enable production to use standard resource limits (100m-2000m CPU, 1500Mi-2Gi memory)
This brings maestro into full compliance with Oracle migration requirements and
implements the standardized affinity/resources pattern consistent with in-factory and duc.
- Enhanced logs to include more detailed request body information.
- Standardized log calls to include extra metadata, such as service name, environment, customer name, user ID, HTTP method, and request path.
- Improved error logging to provide better debugging insights.
Example log format:
EXTRA[{"service":{"name":"maestro","environment":"stg"},"customer_name":"dadosfera","user_id":"047824d8-0021-709a-bf15-39b8dd069aaf","method":"POST","path":"/catalog/register-dataset"}]
Related tickets or issues: [TECN-9441]
As part of our hybrid approach, we need to allow dataset cataloging from outside our cloud environment.
Instead of relying on our GRPC APIs, we leveraged an existing REST API to handle this process.
- Updated Maestro to call Nimbus directly for dataset registration.
- Modified protospack to generate the necessary TypeScript interfaces.
- Added the `/catalog/register-dataset` endpoint to the API.
This change enables seamless dataset registration from external sources while maintaining compatibility with our existing infrastructure.
Related tickets or issues: [TECN-9441]
