2d ago (edited) in Technical
Medallion Architecture Best Practices
Anyone has any best practices for using the Medallion Architecture in Fabric (Bronze, Silver and Gold Lakehouses). I am thinking to do the following, but wanted to understand if this is the right approach for implementing Medallion architecture -
  1. Have Bronze Lakehouse which contains all raw data from source in a Workspace. I do not plan to make any transformations in the Bronze one. I plan to keep this in a workspace which contains shortcuts to all other data sources as well in separate lakehouses
  2. Have Silver Lakehouse which does transformations like - Data Quality checks, de-deuplication (if needed), merging of data from different tables and appending data from different tables. Silver lakehouse will only exist in my Dev workspace
  3. Gold Lakehouse - This is where I will apply business rules and manage any changes ot them, add additional columns, etc needed for reporting and dashboarding. This lakehouse will also be connected to the Semantic Model and PBI. I plan to have Gold lakehouse in Dev, Test and Prod workspaces, which I plan to deploy via Deployment pipelines.
3
2 comments
Atindra Bandi
2
Medallion Architecture Best Practices
Learn Microsoft Fabric
skool.com/microsoft-fabric
Helping passionate analysts, data engineers, data scientists (& more) to advance their careers on the Microsoft Fabric platform.
Leaderboard (30-day)
powered by