Configure project, pipeline, or block level settings for limiting the data volume when running code blocks in development
Autoscaling code execution framework for running blocks during pipeline development
Search files, using full-text and natural language, across multiple projects and add code blocks to pipelines without duplicating code
Run no-code data integrations alongside custom code blocks together within a single pipeline
Data syncs are 12-18x faster using optimized concurrent read/writes with high throughput
Change data capture (CDC) with select databases
Create custom data integration sources and destinations without changing source code
Special SQL block connectors for DuckDB, OracleDB, Teradata, StarRocks, Couchbase, etc.
Add-on
Retrieval Augmented Generation (RAG) pipeline builder
Add-on
AI Sidekick for creating pipelines, generating code blocks, and troubleshooting execution errors
Add-on
Upgraded developer experience for building, managing, and monitoring thousands of SQL models
Add-on
Upgraded developer experience for building, managing, and monitoring thousands of SQL models
Add-on
VSCode integration for building data pipelines in Mage using your local editor
Add-on
Multiple Python virtual environments for development and code block execution
Automatically format code styling and run linters to fix syntax
Multi-tenant support with granular access controls and isolated workspaces
Customizable project and pipeline settings per environment
Integrate your existing CI/CD custom build jobs and deployment steps into Mage Pro
Automatically deploy new code changes and data pipelines to different environments
Git Terminal with built-in authentication and shortcuts
Remote repository import tool for easily migrating or syncing existing projects
Manage multiple Mage projects and any other files from a single application
Add-on
Local file edit tracking and version history for restoring changes made in the past
Add-on
Launch multiple environments and manage shared team workspaces
Add-on
Data version control for data pipeline versioning
Add-on
Configure multiple triggers for a single pipeline with different schedules, frequency, conditions, SLAs, retries, and runtime variables
Schedule pipelines to trigger on a regular or custom interval; e.g. hourly, daily, weekly, monthly, CRON expression
Schedule pipelines to start running and complete its execution by a specific date or time of day
Trigger pipelines from code and optionally wait until completion before proceeding
Trigger pipelines based on events from external 3rd party services
Schedule, run, or re-run backfills multiple times across different windows of time
Add-on
Trigger pipelines across different projects and workspaces
Add-on
Share and reuse a single trigger across different pipelines
Add-on
Backfill data with dynamically generated configurations and variables at runtime using custom code blocks
Add-on
Receive alerts through email, OpsGenie, Slack, Teams, Discord, Telegram, Google Chat, PagerDuty, etc.
Data pipeline execution runtime SLA and alerts
Customizable alert notifications and templates, per channel, when pipeline or block run succeeds, is cancelled, or fails
Integrations with major 3rd party and open source tools, including DataDog, Metaplane, New Relic, Sentry, Prometheus, OpenTelemetry
Manage cross-pipeline dependencies and execution flow across every pipeline within a project
Customizable monitoring dashboard
Alerts via email notification fully integrated without the need for external email service providers
Add-on
Monitor upcoming pipeline execution runs from across all schedules, workspaces, and projects
Add-on
Find and browse complete log history using full text search
Add-on
Custom events, metrics, and alert notification rules
Add-on
System level metrics and logs with dashboard charts
Add-on
Data catalog, metadata management, and data lineage
Add-on
Securely store and access sensitive data or credentials using 3rd party or Mage’s built-in secret manager
Granular pipeline and block run data output retention policies
Single sign on (SSO)
User audit trail and logging
Add-on
Custom security network access rules to allow or deny inbound/outbound traffic from/to a user-defined set of IP addresses
Add-on
Virtual Private Network (VPN) for application account sign-in
Add-on
VPN connection and SSL certificate authentication for select databases
Add-on
Dedicated static IP address for development and running pipelines
Add-on
Regional deployment for data processing operations
Add-on
User management and RBAC permission system with fine-grained access controls to create customizable roles and permissions
Add-on
Execute 100,000+ dynamically created block runs concurrently
Stream block output data to dynamically generated blocks without waiting for the upstream parent block to finish executing
Application upgrades and new product features are instantly installed in your environment
Reusable pipeline data output, accessible across multiple pipelines to reduce duplication, optimize compute, and ensure data consistency
Autoscaling orchestration scheduler for maximum pipeline trigger frequency
Automatically and intelligently scale data pipelines, both vertically and horizontally, using predictive analytics and machine learning
Granular block settings for controlling read/write data partitions using output size, number of chunks, and item count
Add-on
Batch generator framework to operate and process 1,000+ gigabytes (GB) of data without running out of memory
Add-on
Spark compute management and resource monitoring
Add-on
Customized GPU accelerated resources for running AI/ML/LLM pipelines
Add-on
Execute data pipelines using fully customized Docker images
Control and execute pipelines from within other pipelines, using webhooks, or via API requests
Run pipelines and configure runtime variables using no-code user interface elements, such as dropdown menus and autocomplete inputs
Install custom Python modules and system libraries on a per-project basis
High throughput API endpoints for integrating Mage Pro with any 3rd party or in-house services
Deploy and run third-party or custom services integrated with your application environment’s cluster
Add-on
Set up custom domains for different environments and workspaces
Add-on
Manage, orchestrate, and configure infrastructure settings and system resources via API endpoints
Add-on
Fully customize platform application behavior by executing code that transforms any API request payload or response
Add-on
Deploy high-performance, low-latency API endpoints for executing blocks and returning output data, such as inference endpoints
Add-on
