Cloud Composer is a managed workflow automation tool that is built on Apache Airflow. Developers use Cloud Composer to author, schedule and monitor software development pipelines across clouds and on-premise data centers.
Within Cloud Composer, a library of connectors and multiple graphical representations are available through one-click deployment. Cloud Composer pipelines are organized as directed acyclic graphs (DAGs) to accomodate Python files and lower the entry barrier to authoring and scheduling workflows. An Airflow DAG contains a DAG definition, operators and operator relationships.
As part of Google Cloud Platform (GCP), Cloud Composer integrates with tools such as BigQuery, Dataflow, Dataproc, Datastore, Cloud Storage, Pub/Sub and Cloud ML Engine, giving users the ability to orchestrate end-to-end GCP workloads.
Benefits and drawbacks of Google Cloud Composer
The interface of Cloud Composer is user friendly and brings the following benefits:
- Setting up the environment is quick and simple.
- It is easy to integrate any required Python libraries.
- Deployment is straightforward with drag-and-drop or programmatic features.
- Workflows are streamlined for users of all experience levels.
- Is built on an open source orchestration tool that allows for frequent updates and upgrades.
However, one of the drawbacks is that Cloud Composer runs on Python 2.7 rather than a more updated version. Additionally, costs can be hard to calculate ahead of time and can unnecessarily accumulate if not monitored.
Google Cloud Composer charges customers based on consumption, ensuring they pay for what they use. The pricing model is broken down as follows: