Modern Data Platform

Data is ingested into a Raw data layer 'As-Is'. Where possible we recommeend using low-code solutions such as FiveTran or Airbyte to reduce the technical complexity and Total Cost of Ownership.

Through stakeholder workshops and data profiling we will create Conceptual and Logical data models that align with your business. Depending on the requirements the data warehouse will typically have a 3-layer architecture, to Standardise, Conform and Present the data for consumption.

An agile data modelling approach known as Business Event and Analysis Modelling (BEAM) will be utilise to capture the data model, initially in the form of a Business Event Matrix:

dbt enables us to encapsulate the transformation logic in a series of data models, providing the following benefits:

• Easy to use for non engineers (shared data knowledge between engineering and non engineering teams)
• Extremely flexible data model (recreate data easily, backfills are easy)
• Transformations are at a data warehouse level, allowing for massively parallel processing
• Built in testing for data quality
• Online, searchable data catalog and lineage
• Templates for data models and macros for automating tasks such as metadata management
• Low learning curve enables analysts to add their own models
• Production run using dbt cloud.

To support analytics and data science use case we will typically :

• Design and build Semantic models that feed Business Intelligence tools and Generative AI models
• Operationalise Machine Learning workflows
• Provide data services for downstream CRM and marketing systems