Happy Friday everyone! It's Will here, just a short message - I'm seeing people make great progress in their Fabric learning and in their career, which is incredible. Lots of people getting new jobs, or getting certified, or building their first PoC in Fabric! Learning new skills is a long journey, so I think it's important to celebrate the small wins you have along the way. Let's use today to pause and reflect on what YOU'VE done recently - no matter how big or small - and celebrate these wins as a community. Post your Fabric Wins of this week (or recent weeks), maybe you: - Completed a lesson or module? - Built something new in Fabric? - Passed a certification? - Landed a new job? Let's hear about it! Let's spread some positive vibes going into the weekend!!💪

Hello Everyone, Is there an out of a box ETL job that can do Change Data Capture (CDC)? if no out of a box exist, how can i implement such job?

Over the weekend I passed the certification test. I spent Friday night watching the videos to make sure I remembered some of the content.

So... this is some exciting news to share. For the last 3 months or so, I've been building a secret new project, called Fabric Dojo. The Private Preview stage is now complete (thanks to the 10+ of you that have been testing it!), so now it's time to share it with the world 🎉🥳 So what exactly is Fabric Dojo? Fabric Dojo is a brand-new community I am launching here on Skool, to train the next (first?) generation of Fabric Analytics Engineers and Data Engineers. The community is 100% hands-on, with new tutorials, projects and challenges each month. Over time, you'll gain the skills and practical experience you need to feel confident and competent building robust analytical systems in Fabric. All with constant support from myself and the community! Want to learn more? I'll be hosting a webinar on Saturday 31st August to tell you all about it. Meeting Link: Teams Link Datetime of the meeting: Los Angeles - Sat, 31 August 2024, 08:00:00 PDT New York - Sat, 31 August 2024, 11:00:00 EDT London - Sat, 31 August 2024, 16:00:00 BST Dubai - Sat, 31 August 2024, 19:00:00 GST Mumbai Sat, 31 August 2024, 20:30:00 IST Sydney Sun, 01 September 2024, 01:00:00 AEST Here's some sneak peek pictures to hopefully get you excited 😉 see you all on Saturday!

Hello everyone, I'm currently developing a data ingestion pipeline for three regions: APAC, UK, and US, each with multiple SQL servers. Each server manages approximately 400 tables, as shown in the attached screenshot. Notes: 1. The table schema is consistent across all servers. 2. There is no need to store data in a specific geography for insights. 3. Unfortunately, I can't use Database Mirroring and SQL Server CDC features due to IT security issues. The goal is to extract delta data from each server, merge it into a Delta table, and perform transformations following a medallion architecture. I'm considering a Metadata-Driven Pipeline in a hybrid approach: 1. Store the metadata of each table in a Lakehouse or Data Warehouse. 2. The data pipeline will reference this metadata table to iterate through each source table, extracting data from each server and storing it in OneLake. 3. Use PySpark notebooks to merge the delta data from all servers into the Delta table. I'm seeking an efficient and scalable way to implement this in the Fabric. I would appreciate your thoughts and comments.