Cloud Engineering
Cloud Data Ingestion & Storage Platform
3-role access · 0 hard-coded credentials
Graduate project · Sept – Dec 2025
- .NET
- Azure SQL
- Cosmos DB
- Azure Storage
- Key Vault
Problem
Data platforms fail in two ways: bad data gets in, or the wrong people get it out. This platform needed ingested data validated, and access role-gated, before any consumer could reach it.
Architecture
A .NET backend fronting three storage layers, each chosen for its data shape: Azure SQL for structured authentication data and relational metadata with enforced schemas and constraints; Cosmos DB for unstructured image data; Azure Storage for access and view audit events. Access flows through role-based control across three roles — user, approver, admin — and ingested data passes validation plus a human-in-the-loop approval step before becoming consumable. Azure Key Vault holds all secrets; no hard-coded keys live in the pipeline.
Ingested data passes through validation and failure detection, then a human-in-the-loop approval gate, before becoming reachable through role-based access control across three roles — user, approver, admin. Structured authentication data and relational metadata are stored in Azure SQL; unstructured image data is stored in Cosmos DB; access and view events are stored in Azure Storage for auditing. Azure Key Vault supplies secrets to the backend with no hard-coded keys.
Key Decisions
- Storage split by data shape: Azure SQL for constrained, queryable relational metadata; Cosmos DB for image binaries.
- Three-role access model (user, approver, admin), with human-in-the-loop approval gating downstream access.
- Access and view events tracked in Azure Storage as first-class, audit-ready data.
- Secrets held in Azure Key Vault from the outset — no hard-coded credentials in the pipeline.
Results
A working ingestion platform with governed, role-based access; validated ingestion with failure detection; an audit trail of access events; and no hard-coded credentials in the codebase.