Hoping I can get some feedback on an issue we are really struggling with. We are trying to work through adding to our DataVault by adding small pieces at a time which works great with hubs and sats, but breaks down when you get to links. We have some idea of the full build but need to work on projects without having the entire picture. How do we handle adding new keys to an existing Link? Is it better just to add a new object? But then what about all the relationship history that was collected?

Join our free monthly expert session, "Coalesce Transformation Talks"! Every third Thursday, we dive deep into best practices, trends, and challenges. Get valuable insights and have your questions answered by our experts. We kick off this new session next Thursday, 21.11., @Deniz Polat & I will be your host! Register now for free: https://scalefr.ee/ivd29u #coalesce #datamanagement

Like a few others in the community, I will be attending Scalefree’s Data Dreamland conference in Hannover September 23rd and 24th! I will be presenting on optimizing data architecture through a hybrid Data Fabric and Data Mesh approach. If you haven't yet, get your tickets here: https://lnkd.in/ebqPAXjw 🎙️Combining Data Fabric and Data Mesh: Advancing Data Management with Hybrid Architecture 📅 Tuesday, September 24th 🕦 11:30 AM

Hi automation experts, Over the last few weeks, I've had the opportunity to work with the automation tool Datavault Builder. It's a new tool for me, but it looks really promising. I would like to give you some insights into some of the features that I particularly liked. 🔑 Finding the business key An important step when developing an integration flow within Data Vault modeling is to define the business keys. The business key has to be unique, as it identifies the object. Sometimes this process is not as easy as it seems. Datavault Builder supports you in this regard with its Data Viewer. The Data Viewer will give visual feedback, whether the chosen combination of columns results in a unique identification. Additionally a Heatmap will support you to find a proper business key. An even faster way to check, if your composed key is unique can be done by using the uniqueness check while creating a new hub load. 🌐 Using the working canvas As your Data Warehouse grows over time, it can be really challenging to keep track of your Data Model. The working canvas of Datavault Builder can be used to display only some parts of the model or also extend the model. By double-clicking onto a Hub, it is possible to load everything related to that element. This interactive canvas enables you to browse through the whole model step by step. 🏁 End-to-end automation Many tools are able to generate the Raw Data Vault automatically, but the challenge is to automate the Business Vault as well. In Datavault Builder’s business objects layer, you can prepare a denormalized output based on the Raw Data Vault. The Business Object generator will take away the work of manually joining the Data Vault elements and generate an as-of-now-view, on top of which the business logic can be applied in the business rules layer. Also PITs are created implicitly when creating a business object of SCD type 2. In this process, the relevant fields of the used objects are automatically added into the PIT table.

There are so many tools out there, but I found this video on YouTube. It did a good job of breaking down the essentials. Here's a quick list of the tools mentioned: - Basics: SQL, Python, Linux, bash scripting, network understanding - Technical Basics: Git, SFTP, PGP - Databases: PostgreSQL, MySQL, MongoDB - Data Platforms: Snowflake, Databricks, BigQuery, Redshift, Azure Synapse Analytics - Orchestration, ETL & Data Pipelines: Airflow, Dagster, SSAS, Azure Data Factory - Cloud: AWS, Azure, GCP - Others: Docker, Kubernetes, Terraform If you had to give advice on where to start, what would that be? What are your favored tools?