Commit Graph
1592 Commits
Author SHA1 Message Date
Rafael 0a5e8001f9 UPDATE: adding platform_api_url 2025-12-16 16:57:25 -03:00
RafaelandClaude Opus 4.5 01c1087e07 FEAT: add schedule limit validation and improve ES update
- Add schedule limit validation against customer's scheduleLimit from DUC
- Improve ES update to only update provided fields
- Mark GET /platform/pipeline/:pipelineId/pipeline_run as READY

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 16:37:33 -03:00
RafaelandClaude Opus 4.5 f7efb757bf FEAT: add schedule limit validation and improve ES update
- Add cron-validation utility with Airflow presets support (@once, @daily, etc.)
- Validate cron expressions against customer scheduleLimit from DUC
- Fetch scheduleLimit via gRPC to ensure up-to-date configuration
- Improve ES updatePipeline to fetch current doc and merge changes
- Only update fields explicitly provided in PATCH request body
- Preserve config.tables when updating config.cron

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 13:56:49 -03:00
RafaelandClaude Opus 4.5 df674dd441 FEAT: add Platform API proxy with Elasticsearch and DynamoDB sync
- Add Platform API module to proxy requests to Platform API service
- Add Elasticsearch service for pipeline catalog sync (CRUD operations)
- Add DynamoDB service for storing pipeline tables/inputs configuration
- Sync pipeline creation/update/delete to Elasticsearch index
- Extract tables from jobs with connector-specific mappings:
  - JDBC: table_name, load_type, column_include_list, incremental_column_name
  - Singer: replication_method -> full_load/incremental
  - S3: same as Singer
- Map connector types to DynamoDB types (jdbc->database, singer->application, s3->file)
- Validate connector type is provided in job input
- Normalize pipeline IDs for Platform API (replace - with _), keep UUIDs for ES

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 12:42:18 -03:00
Rafael e789076ed4 UPDATE: force deployment 2025-12-02 15:29:36 -03:00
RafaelandClaude 8073194604 fix: make Redis TLS configurable via REDIS_TLS env var
- Change TLS check from ENV !== 'local' to REDIS_TLS === 'true'
- Add REDIS_TLS env var to Helm deployment template
- Default to true for OCI Redis, can override for self-hosted K8s

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 15:10:31 -03:00
Marcos Rodrigues Silva 7f58dc7090 Merge pull request #385 from dadosfera/hotfix/import-files
FIX: permissions
v1.79.2
2025-10-22 18:35:35 -03:00
Marcos Rodrigues 43aa379c06 FIX: permissions 2025-10-22 18:29:43 -03:00
Marcos Rodrigues Silva c284f8753c Merge pull request #384 from dadosfera/hotfix/import-files
FIX: pipelines v2 permissions
v1.79.1
2025-10-22 17:48:26 -03:00
Marcos Rodrigues 3a8f2495c4 FIX: pipelines v2 permissions 2025-10-20 18:13:54 -03:00
Marcos Rodrigues Silva af248716ef Merge pull request #383 from dadosfera/beta
Beta
v1.79.0
2025-10-16 18:15:08 -03:00
Marcos Rodrigues Silva a57ad41ad4 Merge pull request #382 from dadosfera/features/roles
Features/roles
v1.79.0-beta.1
2025-10-16 17:21:37 -03:00
Marcos Rodrigues ffddceec3b FEAT: roles 2025-10-16 17:17:01 -03:00
Marcos Rodrigues Silva b04d5bb402 Merge pull request #381 from dadosfera/feature/tableau-dash
UPDATE: default roles and user
v1.75.0-beta.8
2025-09-23 18:10:35 -03:00
Marcos Rodrigues 53df3caf3f UPDATE: default roles and user 2025-09-23 18:09:25 -03:00
Marcos Rodrigues Silva ab2e35b54f Merge pull request #380 from dadosfera/beta
Beta
v1.78.0
2025-09-17 18:11:05 -03:00
Marcos Rodrigues Silva bfa77d8f7b Merge pull request #379 from dadosfera/feature/auth-cookie
Feature/auth cookie
v1.75.0-beta.7
2025-09-16 17:40:10 -03:00
marcos-silva-rodrigues 5002d147ad UPDATE: add all dadosfera domains 2025-09-16 17:33:19 -03:00
Marcos Rodrigues Silva cd55dc0dc4 Merge pull request #378 from dadosfera/hotfix/uptime-ip
UPDATE: uptime internal ip
v1.77.0
2025-09-15 18:04:14 -03:00
marcos-silva-rodrigues f1d56e4c2c UPDATE: uptime internal ip 2025-09-15 10:34:36 -03:00
Marcos Rodrigues Silva bc49f79cb3 Merge pull request #373 from dadosfera/hotfix/oracle
Network policies
v1.76.0
2025-09-09 18:08:28 -03:00
marcos-silva-rodrigues 00163ad894 UPDATE: network policies 2025-09-09 11:17:45 -03:00
Marcos Rodrigues Silva a2fbeb97cc Merge pull request #377 from dadosfera/feature/auth-cookie
UPDATE: test cors domain
v1.75.0-beta.6
2025-09-08 15:25:05 -03:00
marcos-silva-rodrigues 288a796f46 UPDATE: test cors domain 2025-09-08 15:22:26 -03:00
Marcos Rodrigues Silva b5a1e93770 Merge pull request #376 from dadosfera/feature/auth-cookie
UPDATE: cookie path
v1.75.0-beta.5
2025-09-08 10:32:32 -03:00
marcos-silva-rodrigues 1ee49ceab8 UPDATE: cookie path 2025-09-08 10:30:56 -03:00
Marcos Rodrigues Silva 6eaf9cf6d0 Merge pull request #375 from dadosfera/feature/auth-cookie
UDATE: test cookie domain dadosfera.ai
v1.75.0-beta.4
2025-09-08 09:32:21 -03:00
marcos-silva-rodrigues 075ca747df FIX: ghost commit 2025-09-08 09:26:47 -03:00
marcos-silva-rodrigues 9ead4588c1 UDATE: test cookie domain dadosfera.ai 2025-09-08 09:25:14 -03:00
Marcos Rodrigues Silva 92b64bb362 Merge pull request #374 from dadosfera/hotfix/oracle
Hotfix/oracle
v1.75.0-beta.3
2025-09-05 10:25:38 -03:00
marcos-silva-rodrigues cb8798d871 FIX: enable restricted ips 2025-09-05 10:21:11 -03:00
marcos-silva-rodrigues be70f4da09 FIX: disabled network policies 2025-09-04 10:02:20 -03:00
Marcos Rodrigues Silva b436d7de7a Merge pull request #372 from dadosfera/hotfix/oracle
Hotfix/oracle
v1.75.1
2025-09-03 16:49:31 -03:00
Marcos Rodrigues Silva 6780167f2b Merge pull request #371 from dadosfera/hotfix/oracle
FIX: infinite loading when login is incorrect
v1.75.0-beta.2
2025-09-03 16:31:28 -03:00
marcos-silva-rodrigues 2a3ab4228f FIX: infinite loading when login is incorrect 2025-09-03 16:25:46 -03:00
Marcos Rodrigues Silva caeaf62a9d Merge pull request #370 from dadosfera/hotfix/oracle
Hotfix/oracle
v1.75.0-beta.1
2025-09-03 12:21:17 -03:00
marcos-silva-rodrigues 8e605aa361 FIX: affinity and tls config local 2025-09-03 12:19:30 -03:00
Marcos Rodrigues Silva fa4267f54a Merge pull request #369 from dadosfera/hotfix/oracle
UPDATE: oracle vpn
v1.75.0
2025-08-29 10:16:45 -03:00
marcos-silva-rodrigues 47d8ca2760 UPDATE: oracle vpn 2025-08-29 10:12:58 -03:00
Marcos Rodrigues Silva f3c51e5328 Merge pull request #368 from dadosfera/beta
UPDATE: fixed cloud oracle
v1.74.0
2025-08-28 18:54:02 -03:00
marcos-silva-rodrigues dcf70fdaae UPDATE: fixed cloud oracle v1.72.0-beta.16 2025-08-28 18:40:43 -03:00
Marcos Rodrigues Silva 70f663374d Merge pull request #366 from dadosfera/beta
MIgração para oracle
v1.73.0
2025-08-28 18:32:59 -03:00
Marcos Rodrigues Silva 38f2317bb6 Merge pull request #367 from dadosfera/chore/oracle
UPDATE: affinity and log
v1.72.0-beta.15
2025-08-28 17:05:32 -03:00
marcos-silva-rodrigues 31e0ca6c91 UPDATE: affinity and log 2025-08-28 16:52:52 -03:00
Marcos Rodrigues Silva dc063cbf3f Merge pull request #365 from dadosfera/chore/oracle
Chore/oracle
v1.72.0-beta.14
2025-08-27 17:45:34 -03:00
marcos-silva-rodrigues a4c82ae4a8 UPDATE:migration to oracle 2025-08-27 17:38:26 -03:00
Marcos Rodrigues Silva 11a65e11d1 Merge pull request #364 from dadosfera/chore/oracle
UPDATE: redis host
v1.72.0-beta.13
2025-08-27 14:15:27 -03:00
marcos-silva-rodrigues 73d47f0ff5 UPDATE: merge 2025-08-27 14:13:17 -03:00
marcos-silva-rodrigues d36e532c8e UPDATE: redis host 2025-08-27 13:58:31 -03:00
Marcos Rodrigues Silva b2a24e3a49 Merge pull request #363 from dadosfera/feature/auth-session
UPDATE: test subdomain
v1.72.0-beta.12
2025-08-14 17:51:36 -03:00