Central Data Delivery Platform
Architecture and proof of concept for a unified data layer on Microsoft Fabric – transactional and analytical workloads from one source.
About the Project
A company with multiple applications faced a classic problem of distributed system landscapes: Each application had built and managed its own staging layer for ERP data (SAP and other systems). The result was many different formats, inconsistent data and high maintenance effort across distributed teams.
Requirement
The requirement was a central data platform that prepares ERP data once and provides it for an entire ecosystem of applications – for both transactional and analytical workloads. At the same time, cross-cutting analyses on ERP data combined with application data should be enabled.
Architecture
The developed architecture uses Microsoft Fabric as the central platform. Sync processes replicate ERP data to Fabric, where they are prepared nearly-real-time via Materialized Lake Views for both transactional and analytical processes. The Materialized Views enable performant queries without the complexity of classic ETL pipelines for each consumer.
Result
The concept eliminates redundant staging layers: Applications consume directly from the central platform instead of maintaining their own copies. For transactional applications, GraphQL endpoints were provided that allow querying only the actually needed fields – efficient and flexible.
Highlights
- Conception of a central data layer for transactional and analytical workloads
- Nearly-real-time data preparation via Materialized Lake Views
- GraphQL endpoints for flexible field queries
- Elimination of redundant staging layers
Challenges
Alignment of requirements from different consumers. Design of Materialized Views for both access patterns.
Results
Validated architecture for central data delivery platform. Proof of feasibility for nearly-real-time workloads.
Interested?
Are you interested in a similar project or want to learn more? Get in touch – we look forward to hearing from you!
Get in touch