course
Databricks for Data Engineers: advanced techniques
Learn best practices for using the Databricks Platform as a data engineer.
Description
In this training, you build on your foundational Databricks knowledge and develop a data platform using professional best practices within a realistic mock‑up scenario. You gain hands‑on experience with connecting new data sources, configuring catalogs, setting up security, and working with Git and Databricks Asset Bundles. You apply ingestion techniques such as Merge Into, Lakeflow Connect, and streaming ingestion to reliably process data. Throughout the labs, you put each concept into practice, giving you concrete experience with both batch and streaming workloads.
With declarative Lakeflow Pipelines, you transform data and combine sources into meaningful use cases. You also learn how to monitor your environment using system tables and SQL alerts to detect anomalies in production processes early. In addition, you create metric views and lightweight dashboards that clearly communicate results to end users. By the end of the training, you can deliver a full end‑to‑end data flow (from source to dashboard) including alerting and operational visibility.
Learning Goals
 Describe core Databricks workspace and platform concepts. | Remember  |
| Explain how catalogs, schemas, and permissions work in Unity Catalog. | Understand  |
| Apply ingestion techniques such as Merge Into, Lakeflow Connect, and streaming ingestion. | Apply  |
| Implement data contracts, catalog structures, and basic security settings. | Apply |
| Construct declarative Lakeflow Pipelines for transforming and combining datasets. | Apply |
| Analyze ingestion and transformation runs using system tables, logs, and SQL alerts. | Analyze  |
| Produce metric views and dashboards to present operational insights. | Apply |
| Interpret monitoring signals to identify anomalies in batch or streaming workloads. | Understand |
| Implement an end‑to‑end data flow from source ingestion to dashboarding. | Apply |
Prior Knowledge
- Familiarity with the Databricks platform
- Solid understanding of data engineering concepts such as ETL/ELT, data pipelines, and data lakes / data warehouses
Subjects
- Introduction & Databricks Environment
- Ingestion
- Transformation
- Monitoring
- Serving
1. Introduction & Databricks Environment
- Introduction
- Catalog configuration
- Grants & security
- Git integration
- Lab: Load repository, prepare mock-up data platform, use Databricks CLI
2. Ingestion
- Data contracts
- Merge Into
- Connecting a Parquet source
- Lakeflow Connect
- Cluster configuration tuning & serverless
- Lab: Connect Parquet source
- Lab: Connect streaming source
3. Transformation
- Lakeflow Declarative Pipelines
- SQL Alerts
- Lab: Build pipelines based on existing and new sources
4. Monitoring
- System tables
- System dashboarding
- Lab: Review pipeline results
5. Serving
- Metric Views
- Dashboarding
- Genie Space
- Power BI Desktop
- Lab: Build metric views to support dashboards
- Lab: Create dashboards and monitor end‑to‑end operations
Schedule
| Start date | Duration | Location | |
|---|---|---|---|
October 5, 2026October 6, 2026 | 2 days | Veenendaal / Remote This is a hybrid training and can be followed remotely. More information Veenendaal / Remote This is a hybrid training and can be followed remotely. More information | Sign up |
All courses can also be conducted within your organization as customized or incompany training.
Our training advisors are happy to help you provide personal advice or find Incompany training within your organization.
Prior knowledge courses
"Trainer who knows his profession!"Marc
-
Hoge waardering
-
Praktijkgerichte trainingen
-
Gecertificeerde trainers
-
Eigen docenten