I am trying to use one of the visualization tool called atoti. Their widget needs to be launched from a jupyter lab after installing an extension: atoti[jupyter-lab]. Is there any way to connect from jupyter-lab to the spark engine (in a similar way to what is being done in VS code), that would allow me to query my delta tables using the Fabric Compute resources and then build my atoti visuals in the lab itself? Thanks.

Hey Fabricator! It’s another tuesday and WE (aka the whole community! ) are curious to know : - What are you working on? - Any blockers or challenges you’re up against? These could be in a project at work, a pet project, or something in your learning journey. There’s a good chance someone in the community has tackled something similar or has a fresh perspective. So don’t hesitate to share in the comments, or feel free to make a separate post if you’d like more focused feedback. No blocker is too small to ask for advice on, so FEEL FREE TO SHARE ! Looking forward to hearing what you are up to!

Team, I'm running 1 notebook processing multiple tables from bronze bringing them into 1 silver table. The processing takes up half of F128 capacity during the 30 min it takes to execute. This seems extremely high to me, any thoughts? Additionally I'm seeing copilot requests on the data but not using any copilot features that I am aware of? Any ideas are appreciated

I know there are decision guides made on data ingestion and datastorages. But i find it quite difficult to find a comparison of data transformation methods, and how these compare with performance and cost. Does anyone have any insight on this to share? :)

Hi Guys, Just wanted to ask about Partitioning with Direct Lake. I already have a very large delta table, roughly 60 million rows. Every hour I am appending data to this table using a notebook. I have partitioned this table using year and month (so roughly 84 partitions). I assume the benefit of partition is that the append is easier and the optimize function doesn't have to join up the 60 million rows but rather the append files inside of the latest year+month combination. However when I go to the Microsoft guide it tells me that I should avoid using partitions if my goal is to use a delta table for a semantic model (which it is): Microsoft Reference: https://learn.microsoft.com/en-us/fabric/get-started/direct-lake-understand-storage#table-partitioning Important If the main purpose of a Delta table is to serve as a data source for semantic models (and secondarily, other query workloads), it's usually better to avoid partitioning in preference for optimizing the load of columns into memory. Questions: 1. Should I avoid using the partition? 2. What examples are there of why we need to partition? Any help will be much appreciated. Thanks