Training Taster Cloud Composer
Date: Thursday, June 24, 2021
Time: 14:00 – 17:00 CET (5 AM – 8 AM PST | 8 AM – 11 AM EST | 5:30 PM – 8:30 PM IST)
Trainer: Bas Harenslak
Cost: 50 USD
Participants receive a certificate of attendance
In this training we will focus on GCP Cloud Composer, Google’s managed Apache Airflow service. Through a series of examples and small exercises we learn how to best develop data pipelines with Apache Airflow and connect several GCP services together.
What you’ll learn
- The basic structure of Apache Airflow data pipelines
- How to deploy and run a data pipeline on GCP Cloud Composer
- When to (not) use GCP Cloud Composer
About Google Cloud Composer
Benefits
Fully managed workflow orchestration
Cloud Composer’s managed nature and Apache Airflow compatibility allows you to focus on authoring, scheduling, and monitoring your workflows as opposed to provisioning resources.
Integrates with other Google Cloud products
End-to-end integration with Google Cloud products including BigQuery, Dataflow, Dataproc, Datastore, Cloud Storage, Pub/Sub, and AI Platform gives users the freedom to fully orchestrate their pipeline.
Key features
Hybrid and multi-cloud
Ease your transition to the cloud or maintain a hybrid data environment by orchestrating workflows that cross between on-premises and the public cloud. Create workflows that connect data, processing, and services across clouds to give you a unified data environment.
Open source
Cloud Composer is built upon Apache Airflow, giving users freedom from lock-in and portability. This open source project, which Google is contributing back into, provides freedom from lock-in for customers as well as integration with a broad number of platforms, which will only expand as the Airflow community grows.
Easy orchestration
Cloud Composer pipelines are configured as directed acyclic graphs (DAGs) using Python, making it easy for any user. One-click deployment yields instant access to a rich library of connectors and multiple graphical representations of your workflow in action, making troubleshooting easy. Automatic synchronization of your directed acyclic graphs ensures your jobs stay on schedule.
About the Trainer
Bas Harenslak
DATA ENGINEER
Bas is a Data Engineer at GoDataDriven and co-author of the book “Data pipelines with Airflow” (Manning Publishers).
Bas finished a Master’s in Computer Science cum laude. His goal is to build zero maintenance data-driven products for real-world solutions. His current interests lie in the cloud and containerised world as he believes it brings the fastest end user value and best software. He has an open source and start-up mentality and is always looking for challenging ways to learn and improve.
Clients include: Booking.com, Unilever Food Solutions, Knab, Yolt (by ING), ANWB, and the Dutch Police
