Databricks for Data Engineers: advanced techniques

Name: Databricks for Data Engineers: advanced techniques
Price: 2685 EUR

Learn best practices for using the Databricks Platform as a data engineer.

March 22, 2027

-Veenendaal / Remote

-View more dates

3 days

2685 (excl. VAT)

Description

In this training, you build on your foundational Databricks knowledge and develop a data platform using professional best practices within a realistic mock‑up scenario. You gain hands‑on experience with connecting new data sources, configuring catalogs, setting up security, and working with Git and Databricks Asset Bundles. You apply ingestion techniques such as Merge Into, Lakeflow Connect, and streaming ingestion to reliably process data. Throughout the labs, you put each concept into practice, giving you concrete experience with both batch and streaming workloads.

With declarative Lakeflow Pipelines, you transform data and combine sources into meaningful use cases. You also learn how to monitor your environment using system tables and SQL alerts to detect anomalies in production processes early. In addition, you create metric views and lightweight dashboards that clearly communicate results to end users. By the end of the training, you can deliver a full end‑to‑end data flow (from source to dashboard) including alerting and operational visibility.

Learning Goals

-Describe core Databricks workspace concepts, platform architecture, and compute options.
+Remember
-Explain how catalogs, schemas, and permissions work in Unity Catalog.
+Understand
-Apply ingestion techniques such as Merge Into, Auto Loader, and streaming ingestion.
+Apply
-Implement data contracts and catalog structures to enforce data governance.
+Apply
-Construct and validate declarative Lakeflow Pipelines for transforming and combining datasets.
+Apply
-Analyze ingestion and transformation runs using system tables, logs, and SQL alerts.
+Analyze
-Produce metric views, dashboards, and data products — including conversational analytics — to deliver operational insights.
+Apply
-Interpret monitoring signals to identify anomalies in batch or streaming workloads.
+Understand
-Implement an end‑to‑end data flow from source ingestion to dashboarding.
+Apply

For the above learning goals we use Bloom's Taxonomy

Prior Knowledge

Familiarity with the Databricks platform
Solid understanding of data engineering concepts such as ETL/ELT, data pipelines, and data lakes / data warehouses
Experience working with Apache Spark

Subjects

Databricks Platform
Ingestion
Declarative Pipelines
Transformation & Metric Views
Serving
Monitoring

1. Databricks Platform

Unity Catalog and schemas
Catalog configuration
Grants & security
Git integration and Databricks Asset Bundles
Lab 1: Platform Setup

2. Ingestion

Ingestion types and patterns
Data contracts
Auto Loader
Schema evolution
Lab 2: Machine Masterdata Ingestion

3. Declarative Pipelines

Lakeflow Connect
Cluster configuration tuning & serverless
Lakeflow Declarative Pipelines
Structured Streaming
Data quality expectations
Lab 3: Streaming, Declarative Pipelines, and Data Quality

4. Transformation & Metric Views

Advanced pipeline patterns
Data quality checks and the testing duality
Metric Views as a semantic layer
Data Masking and Filtering
Lab 4: Business Logic, Metric Views, and Data Governance

5. Serving

Data product thinking
AI/BI Dashboarding
Genie Space & AI Agents
Serving External Tools
Lab 5: Serving Your Data

6. Monitoring

System tables
System dashboards
SQL Alerts
Lab 6: Monitoring Your Data Platform

Start date	Duration	Location
March 22, 2027March 23, 2027March 24, 2027	3 days	Veenendaal / Remote This is a hybrid training and can be followed remotely. More information Veenendaal / Remote This is a hybrid training and can be followed remotely. More information Veenendaal / Remote This is a hybrid training and can be followed remotely. More information	Sign up