5.1 Introduction to triggers
Mage triggers are the orchestration mechanism that determines when and how your data pipelines execute. They provide automated execution capabilities that eliminate the need for manual pipeline runs, enabling reliable and scalable data operations. Triggers act as the scheduling and event-driven backbone of your data infrastructure.
Mage offers several trigger types to accommodate different execution patterns and business requirements:
Schedule triggers - Execute pipelines based on time-based schedules (cron expressions)
API triggers - Execute pipelines via HTTP requests for real-time processing
Streaming triggers - Execute streaming pipelines for continuous data processing
Triggers provide essential benefits for production data workflows:
Automation: Eliminate manual intervention and reduce operational overhead
Reliability: Ensure consistent execution timing and error handling
Scalability: Handle multiple pipeline executions across different schedules
Flexibility: Support various execution patterns from batch to real-time processing
Monitoring: Track execution history and performance metrics
| Trigger Type | Use Case | Execution Pattern | Complexity |
|---|---|---|---|
| Schedule | Daily ETL, Reports | Time-based | Low |
| API | Real-time processing | On-demand | Medium |
| Streaming | Continuous Processing | Real-time | High |
Understanding triggers is fundamental to building production-ready data pipelines that operate independently and scale with your organization's needs.