Like a few others in the community, I will be attending Scalefree’s Data Dreamland conference in Hannover September 23rd and 24th! I will be presenting on optimizing data architecture through a hybrid Data Fabric and Data Mesh approach. If you haven't yet, get your tickets here: https://lnkd.in/ebqPAXjw 🎙️Combining Data Fabric and Data Mesh: Advancing Data Management with Hybrid Architecture 📅 Tuesday, September 24th 🕦 11:30 AM

Hi automation experts, Over the last few weeks, I've had the opportunity to work with the automation tool Datavault Builder. It's a new tool for me, but it looks really promising. I would like to give you some insights into some of the features that I particularly liked. 🔑 Finding the business key An important step when developing an integration flow within Data Vault modeling is to define the business keys. The business key has to be unique, as it identifies the object. Sometimes this process is not as easy as it seems. Datavault Builder supports you in this regard with its Data Viewer. The Data Viewer will give visual feedback, whether the chosen combination of columns results in a unique identification. Additionally a Heatmap will support you to find a proper business key. An even faster way to check, if your composed key is unique can be done by using the uniqueness check while creating a new hub load. 🌐 Using the working canvas As your Data Warehouse grows over time, it can be really challenging to keep track of your Data Model. The working canvas of Datavault Builder can be used to display only some parts of the model or also extend the model. By double-clicking onto a Hub, it is possible to load everything related to that element. This interactive canvas enables you to browse through the whole model step by step. 🏁 End-to-end automation Many tools are able to generate the Raw Data Vault automatically, but the challenge is to automate the Business Vault as well. In Datavault Builder’s business objects layer, you can prepare a denormalized output based on the Raw Data Vault. The Business Object generator will take away the work of manually joining the Data Vault elements and generate an as-of-now-view, on top of which the business logic can be applied in the business rules layer. Also PITs are created implicitly when creating a business object of SCD type 2. In this process, the relevant fields of the used objects are automatically added into the PIT table.

There are so many tools out there, but I found this video on YouTube. It did a good job of breaking down the essentials. Here's a quick list of the tools mentioned: - Basics: SQL, Python, Linux, bash scripting, network understanding - Technical Basics: Git, SFTP, PGP - Databases: PostgreSQL, MySQL, MongoDB - Data Platforms: Snowflake, Databricks, BigQuery, Redshift, Azure Synapse Analytics - Orchestration, ETL & Data Pipelines: Airflow, Dagster, SSAS, Azure Data Factory - Cloud: AWS, Azure, GCP - Others: Docker, Kubernetes, Terraform If you had to give advice on where to start, what would that be? What are your favored tools?

Check out this audio version of the top story from DataPro.news https://www.datapro.news/p/adapting-data-governance-age-llms We discuss the key considerations in Data Governance as Enterprises adopt LLM's. Let me know anything you liked!