oabrivard
/
mdp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
master
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'master'
${ noResults }
mdp/Graphics/Medallion_Architecture_Diag...

388 lines
31 KiB
XML
Raw Permalink Blame History Unescape Escape

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1580 900" font-family="Arial, Helvetica, sans-serif">
<defs>
<linearGradient id="gBronze" x1="0" y1="0" x2="0" y2="1"><stop offset="0%" stop-color="#EDD5A8"/><stop offset="100%" stop-color="#D4A76A"/></linearGradient>
<linearGradient id="gSilver" x1="0" y1="0" x2="0" y2="1"><stop offset="0%" stop-color="#D8DDE3"/><stop offset="100%" stop-color="#B8C0CC"/></linearGradient>
<linearGradient id="gGold" x1="0" y1="0" x2="0" y2="1"><stop offset="0%" stop-color="#F7E9A6"/><stop offset="100%" stop-color="#E0C84A"/></linearGradient>
<linearGradient id="gGreen" x1="0" y1="0" x2="1" y2="0"><stop offset="0%" stop-color="#006B3F"/><stop offset="100%" stop-color="#00944F"/></linearGradient>
<linearGradient id="gTeal" x1="0" y1="0" x2="1" y2="0"><stop offset="0%" stop-color="#00706E"/><stop offset="100%" stop-color="#00A09D"/></linearGradient>
<filter id="sh"><feDropShadow dx="1" dy="2" stdDeviation="2.5" flood-opacity="0.13"/></filter>
<marker id="ar" viewBox="0 0 10 7" refX="9" refY="3.5" markerWidth="9" markerHeight="7" orient="auto"><path d="M0 0 L10 3.5 L0 7z" fill="#444"/></marker>
<marker id="arG" viewBox="0 0 10 7" refX="9" refY="3.5" markerWidth="9" markerHeight="7" orient="auto"><path d="M0 0 L10 3.5 L0 7z" fill="#006B3F"/></marker>
<marker id="arP" viewBox="0 0 10 7" refX="9" refY="3.5" markerWidth="9" markerHeight="7" orient="auto"><path d="M0 0 L10 3.5 L0 7z" fill="#7B1FA2"/></marker>
</defs>
<!-- Background -->
<rect width="1580" height="900" fill="#FAFBFC" rx="6"/>
<!-- Title bar -->
<rect x="0" y="0" width="1580" height="52" fill="#004D2C" rx="6"/>
<rect x="0" y="20" width="1580" height="32" fill="#004D2C"/>
<text x="790" y="33" text-anchor="middle" font-size="21" font-weight="bold" fill="white">Medallion Architecture — Physical Deployment on Azure Infrastructure</text>
<!-- ═══════════════════════════════════════════════════════ -->
<!-- ROW 1: SOURCE → INGESTION → PRE-BRONZE DQ → BRONZE -->
<!-- ═══════════════════════════════════════════════════════ -->
<!-- SOURCE SYSTEMS -->
<rect x="30" y="80" width="130" height="250" rx="8" fill="#F5F5F5" stroke="#AAA" stroke-width="1.2" filter="url(#sh)"/>
<rect x="30" y="80" width="130" height="30" rx="8" fill="#757575"/>
<rect x="30" y="98" width="130" height="12" fill="#757575"/>
<text x="95" y="100" text-anchor="middle" font-size="13" font-weight="bold" fill="white">Source Systems</text>
<text x="95" y="132" text-anchor="middle" font-size="11" fill="#555">DB2 · Oracle</text>
<text x="95" y="150" text-anchor="middle" font-size="11" fill="#555">SQL Server</text>
<text x="95" y="168" text-anchor="middle" font-size="11" fill="#555">Mainframe</text>
<text x="95" y="186" text-anchor="middle" font-size="11" fill="#555">SFTP / APIs</text>
<text x="95" y="204" text-anchor="middle" font-size="11" fill="#555">Event Streams</text>
<rect x="42" y="222" width="106" height="26" rx="4" fill="#E0E0E0" stroke="#999" stroke-width="0.5"/>
<text x="95" y="240" text-anchor="middle" font-size="10" fill="#555">ExpressRoute</text>
<text x="95" y="268" text-anchor="middle" font-size="9" fill="#888">via Hub VNet</text>
<text x="95" y="282" text-anchor="middle" font-size="9" fill="#888">+ Azure Firewall</text>
<text x="95" y="318" text-anchor="middle" font-size="9" fill="#AAA">On-premises</text>
<!-- Arrow Sources → Ingestion -->
<line x1="160" y1="200" x2="192" y2="200" stroke="#444" stroke-width="2" marker-end="url(#ar)"/>
<!-- INGESTION LAYER -->
<rect x="195" y="80" width="180" height="250" rx="8" fill="#E8F5E9" stroke="#43A047" stroke-width="1.3" filter="url(#sh)"/>
<rect x="195" y="80" width="180" height="30" rx="8" fill="#43A047"/>
<rect x="195" y="98" width="180" height="12" fill="#43A047"/>
<text x="285" y="100" text-anchor="middle" font-size="13" font-weight="bold" fill="white">Ingestion Layer</text>
<rect x="208" y="120" width="154" height="36" rx="5" fill="white" stroke="#66BB6A" stroke-width="1"/>
<text x="285" y="138" text-anchor="middle" font-size="11" font-weight="bold" fill="#333">Azure Data Factory</text>
<text x="285" y="150" text-anchor="middle" font-size="9" fill="#666">Batch · CDC · Copy Activity</text>
<rect x="208" y="164" width="154" height="36" rx="5" fill="white" stroke="#66BB6A" stroke-width="1"/>
<text x="285" y="182" text-anchor="middle" font-size="11" font-weight="bold" fill="#333">Auto Loader</text>
<text x="285" y="194" text-anchor="middle" font-size="9" fill="#666">cloudFiles · Schema inference</text>
<rect x="208" y="208" width="154" height="36" rx="5" fill="white" stroke="#66BB6A" stroke-width="1"/>
<text x="285" y="226" text-anchor="middle" font-size="11" font-weight="bold" fill="#333">Azure Event Hub</text>
<text x="285" y="238" text-anchor="middle" font-size="9" fill="#666">Streaming · Fraud events</text>
<rect x="208" y="256" width="154" height="28" rx="4" fill="#C8E6C9"/>
<text x="285" y="274" text-anchor="middle" font-size="9" font-weight="bold" fill="#2E7D32">stadlsstagingprod</text>
<text x="285" y="286" text-anchor="middle" font-size="8" fill="#555">ADLS Staging Zone</text>
<rect x="208" y="296" width="154" height="22" rx="3" fill="#F1F8E9"/>
<text x="285" y="312" text-anchor="middle" font-size="9" fill="#33691E">Orch: Databricks Workflows / ADF</text>
<!-- Arrow Ingestion → DQ Gate -->
<line x1="375" y1="200" x2="407" y2="200" stroke="#444" stroke-width="2" marker-end="url(#ar)"/>
<!-- PRE-BRONZE DQ GATE -->
<rect x="410" y="110" width="118" height="180" rx="8" fill="#FFF8E1" stroke="#FFA000" stroke-width="1.3" filter="url(#sh)"/>
<rect x="410" y="110" width="118" height="30" rx="8" fill="#FF8F00"/>
<rect x="410" y="128" width="118" height="12" fill="#FF8F00"/>
<text x="469" y="130" text-anchor="middle" font-size="12" font-weight="bold" fill="white">DQ Gate</text>
<text x="469" y="160" text-anchor="middle" font-size="11" font-weight="bold" fill="#E65100">Purview</text>
<text x="469" y="176" text-anchor="middle" font-size="11" font-weight="bold" fill="#E65100">Data Quality</text>
<text x="469" y="198" text-anchor="middle" font-size="10" fill="#555">Sampling-based</text>
<text x="469" y="213" text-anchor="middle" font-size="10" fill="#555">assessment</text>
<line x1="424" y1="226" x2="514" y2="226" stroke="#FFB74D" stroke-width="0.5"/>
<text x="469" y="244" text-anchor="middle" font-size="10" fill="#2E7D32">✓ Pass → Bronze</text>
<text x="469" y="262" text-anchor="middle" font-size="10" fill="#C62828">✗ Fail → Quarantine</text>
<text x="469" y="282" text-anchor="middle" font-size="9" fill="#888">Tier 1 DQ</text>
<!-- Arrow DQ → Bronze -->
<line x1="528" y1="200" x2="562" y2="200" stroke="#444" stroke-width="2" marker-end="url(#ar)"/>
<!-- ═══════════════════════════════════════════════════════ -->
<!-- MEDALLION LAYERS (center, three tall boxes) -->
<!-- ═══════════════════════════════════════════════════════ -->
<!-- BRONZE -->
<rect x="565" y="80" width="200" height="250" rx="10" fill="url(#gBronze)" stroke="#B8860B" stroke-width="1.8" filter="url(#sh)"/>
<text x="665" y="110" text-anchor="middle" font-size="20" font-weight="bold" fill="#5D3A1A">BRONZE</text>
<text x="665" y="130" text-anchor="middle" font-size="11" fill="#6D4C2A">Raw Immutable Store</text>
<rect x="578" y="144" width="174" height="42" rx="5" fill="white" stroke="#B8860B" stroke-width="1" opacity="0.92"/>
<text x="665" y="163" text-anchor="middle" font-size="12" font-weight="bold" fill="#333">stadlsbronzeprod</text>
<text x="665" y="179" text-anchor="middle" font-size="9" fill="#666">ADLS Gen2 · Delta Lake · Hot tier</text>
<rect x="578" y="194" width="174" height="30" rx="4" fill="#E8F5E9" stroke="#66BB6A" stroke-width="0.8"/>
<text x="665" y="209" text-anchor="middle" font-size="10" font-weight="bold" fill="#2E7D32">Unity Catalog</text>
<text x="665" y="220" text-anchor="middle" font-size="8.5" fill="#555">bronze_*.* · Data eng roles only</text>
<text x="665" y="246" text-anchor="middle" font-size="10" fill="#5D3A1A">Append-only · 90-day Time Travel</text>
<text x="665" y="262" text-anchor="middle" font-size="10" fill="#5D3A1A">mergeSchema · Schema-on-read</text>
<text x="665" y="280" text-anchor="middle" font-size="9" fill="#8D6E2F">/bronze/{source_system}/{entity}/</text>
<text x="665" y="318" text-anchor="middle" font-size="8" fill="#888">PE: pe-stadlsbronzeprod-dfs</text>
<!-- DLT Bronze→Silver transition -->
<rect x="590" y="342" width="150" height="46" rx="6" fill="url(#gGreen)" filter="url(#sh)"/>
<text x="665" y="362" text-anchor="middle" font-size="12" font-weight="bold" fill="white">Databricks DLT</text>
<text x="665" y="378" text-anchor="middle" font-size="9" fill="#C8E6C9">Cleanse · Conform · Deduplicate</text>
<!-- DLT quality label -->
<text x="755" y="358" font-size="9" font-weight="bold" fill="#006B3F">DLT Expectations +</text>
<text x="755" y="372" font-size="9" font-weight="bold" fill="#006B3F">Great Expectations</text>
<text x="755" y="384" font-size="9" fill="#888">(Tier 2 DQ)</text>
<!-- Arrow Bronze → DLT → Silver -->
<line x1="665" y1="330" x2="665" y2="340" stroke="#006B3F" stroke-width="2" marker-end="url(#arG)"/>
<line x1="665" y1="388" x2="665" y2="406" stroke="#006B3F" stroke-width="2" marker-end="url(#arG)"/>
<!-- SILVER -->
<rect x="565" y="408" width="200" height="210" rx="10" fill="url(#gSilver)" stroke="#78909C" stroke-width="1.8" filter="url(#sh)"/>
<text x="665" y="436" text-anchor="middle" font-size="20" font-weight="bold" fill="#37474F">SILVER</text>
<text x="665" y="456" text-anchor="middle" font-size="11" fill="#546E7A">Curated &amp; Conformed</text>
<rect x="578" y="468" width="174" height="42" rx="5" fill="white" stroke="#78909C" stroke-width="1" opacity="0.92"/>
<text x="665" y="487" text-anchor="middle" font-size="12" font-weight="bold" fill="#333">stadlssilverprod</text>
<text x="665" y="503" text-anchor="middle" font-size="9" fill="#666">ADLS Gen2 · Delta Lake · Hot tier</text>
<rect x="578" y="518" width="174" height="30" rx="4" fill="#E8F5E9" stroke="#66BB6A" stroke-width="0.8"/>
<text x="665" y="533" text-anchor="middle" font-size="10" font-weight="bold" fill="#2E7D32">Unity Catalog</text>
<text x="665" y="544" text-anchor="middle" font-size="8.5" fill="#555">silver_*.* · RLS + CLS enforced</text>
<text x="665" y="568" text-anchor="middle" font-size="10" fill="#37474F">Business keys · SCD Type 2</text>
<text x="665" y="584" text-anchor="middle" font-size="10" fill="#37474F">30-day Time Travel · Strict schema</text>
<text x="665" y="604" text-anchor="middle" font-size="9" fill="#546E7A">/silver/{domain}/{entity}/</text>
<!-- DLT Silver→Gold transition -->
<rect x="590" y="630" width="150" height="46" rx="6" fill="url(#gGreen)" filter="url(#sh)"/>
<text x="665" y="650" text-anchor="middle" font-size="12" font-weight="bold" fill="white">DLT / SQL</text>
<text x="665" y="666" text-anchor="middle" font-size="9" fill="#C8E6C9">Aggregate · Denormalize · KPIs</text>
<text x="755" y="646" font-size="9" font-weight="bold" fill="#006B3F">SLA Gate: ≥99.5%</text>
<text x="755" y="660" font-size="9" font-weight="bold" fill="#006B3F">CDE completeness</text>
<text x="755" y="672" font-size="9" fill="#888">(Tier 3 DQ)</text>
<line x1="665" y1="618" x2="665" y2="628" stroke="#006B3F" stroke-width="2" marker-end="url(#arG)"/>
<line x1="665" y1="676" x2="665" y2="696" stroke="#006B3F" stroke-width="2" marker-end="url(#arG)"/>
<!-- GOLD -->
<rect x="565" y="698" width="200" height="140" rx="10" fill="url(#gGold)" stroke="#C6A300" stroke-width="1.8" filter="url(#sh)"/>
<text x="665" y="726" text-anchor="middle" font-size="20" font-weight="bold" fill="#5D4E00">GOLD</text>
<text x="665" y="746" text-anchor="middle" font-size="11" fill="#6D5A00">Business-Ready Data Products</text>
<rect x="578" y="758" width="174" height="42" rx="5" fill="white" stroke="#C6A300" stroke-width="1" opacity="0.92"/>
<text x="665" y="777" text-anchor="middle" font-size="12" font-weight="bold" fill="#333">stadlsgoldprod</text>
<text x="665" y="793" text-anchor="middle" font-size="9" fill="#666">ADLS Gen2 · Delta Lake · Hot · RA-GRS</text>
<text x="665" y="818" text-anchor="middle" font-size="10" fill="#5D4E00">Star schemas · Feature tables · KPIs</text>
<text x="665" y="834" text-anchor="middle" font-size="9" fill="#6D5A00">Data contracts · LIQUID clustering</text>
<!-- ═══════════════════════════════════════════════════════ -->
<!-- DATABRICKS COMPUTE (left panel, vertical) -->
<!-- ═══════════════════════════════════════════════════════ -->
<rect x="195" y="360" width="170" height="480" rx="8" fill="url(#gGreen)" filter="url(#sh)"/>
<text x="280" y="388" text-anchor="middle" font-size="15" font-weight="bold" fill="white">Databricks Compute</text>
<text x="280" y="406" text-anchor="middle" font-size="10" fill="#A5D6A7">VNet-Injected | No Public IP</text>
<rect x="206" y="420" width="148" height="50" rx="5" fill="white" opacity="0.92"/>
<text x="280" y="438" text-anchor="middle" font-size="11" font-weight="bold" fill="#006B3F">dbw-data-eng-prod</text>
<text x="280" y="452" text-anchor="middle" font-size="9" fill="#555">DLT pipelines · DS4_v2</text>
<text x="280" y="464" text-anchor="middle" font-size="9" fill="#555">Auto-term 30m · Spot 60/40</text>
<rect x="206" y="478" width="148" height="50" rx="5" fill="white" opacity="0.92"/>
<text x="280" y="496" text-anchor="middle" font-size="11" font-weight="bold" fill="#006B3F">dbw-analytics-prod</text>
<text x="280" y="510" text-anchor="middle" font-size="9" fill="#555">Serverless SQL Warehouses</text>
<text x="280" y="522" text-anchor="middle" font-size="9" fill="#555">2–16 DBU · Auto-suspend</text>
<rect x="206" y="536" width="148" height="50" rx="5" fill="white" opacity="0.92"/>
<text x="280" y="554" text-anchor="middle" font-size="11" font-weight="bold" fill="#006B3F">dbw-mlops-prod</text>
<text x="280" y="568" text-anchor="middle" font-size="9" fill="#555">MLflow · Feature Store</text>
<text x="280" y="580" text-anchor="middle" font-size="9" fill="#555">GPU NC6s_v3 · Model Serving</text>
<rect x="206" y="594" width="148" height="40" rx="5" fill="white" opacity="0.92"/>
<text x="280" y="612" text-anchor="middle" font-size="11" font-weight="bold" fill="#006B3F">dbw-sandbox</text>
<text x="280" y="626" text-anchor="middle" font-size="9" fill="#555">Read-only Silver/Gold</text>
<!-- Subnet details -->
<rect x="206" y="646" width="148" height="60" rx="5" fill="#A5D6A7" opacity="0.5"/>
<text x="280" y="664" text-anchor="middle" font-size="10" font-weight="bold" fill="white">Subnets /22 each</text>
<text x="280" y="680" text-anchor="middle" font-size="9" fill="white">snet-dbx-host (10.10.0.0/22)</text>
<text x="280" y="694" text-anchor="middle" font-size="9" fill="white">snet-dbx-container (10.10.4.0/22)</text>
<!-- Security details -->
<text x="280" y="722" text-anchor="middle" font-size="9" fill="#C8E6C9">Cluster policies · CMK encryption</text>
<text x="280" y="738" text-anchor="middle" font-size="9" fill="#C8E6C9">PATs disabled · OAuth M2M</text>
<text x="280" y="754" text-anchor="middle" font-size="9" fill="#C8E6C9">Secret scopes → Key Vault</text>
<text x="280" y="770" text-anchor="middle" font-size="9" fill="#C8E6C9">Secure cluster connectivity</text>
<!-- SQL Warehouses -->
<rect x="206" y="784" width="148" height="44" rx="5" fill="white" opacity="0.92"/>
<text x="280" y="802" text-anchor="middle" font-size="10" font-weight="bold" fill="#006B3F">SQL Warehouses</text>
<text x="280" y="816" text-anchor="middle" font-size="8.5" fill="#555">sqlwh-bi-serving · sqlwh-analytics</text>
<text x="280" y="828" text-anchor="middle" font-size="8.5" fill="#555">snet-sqlwarehouse (10.10.9.0/24)</text>
<!-- ═══════════════════════════════════════════════════════ -->
<!-- CONSUMPTION ENDPOINTS (right side) -->
<!-- ═══════════════════════════════════════════════════════ -->
<!-- Arrows from Gold to endpoints -->
<!-- Gold → Power BI -->
<path d="M 765 730 L 810 730 Q 830 730 830 720 L 830 120 Q 830 108 850 108 L 870 108" stroke="#7B1FA2" stroke-width="2" fill="none" marker-end="url(#arP)"/>
<text x="840" y="425" font-size="9" font-weight="bold" fill="#7B1FA2" transform="rotate(-90,840,425)">Zero-copy OneLake Shortcuts</text>
<!-- Power BI / Fabric -->
<rect x="872" y="80" width="240" height="105" rx="8" fill="#F3E5F5" stroke="#7B1FA2" stroke-width="1.4" filter="url(#sh)"/>
<rect x="872" y="80" width="240" height="30" rx="8" fill="#7B1FA2"/>
<rect x="872" y="98" width="240" height="12" fill="#7B1FA2"/>
<text x="992" y="100" text-anchor="middle" font-size="13" font-weight="bold" fill="white">6a Power BI via Fabric</text>
<text x="992" y="126" text-anchor="middle" font-size="11" font-weight="bold" fill="#4A148C">F64 Capacity (Prod)</text>
<text x="992" y="144" text-anchor="middle" font-size="10" fill="#555">Direct Lake semantic models</text>
<text x="992" y="160" text-anchor="middle" font-size="10" fill="#555">55,000 users · sub-data-fabric</text>
<text x="992" y="176" text-anchor="middle" font-size="9" fill="#888">Managed PaaS · Auto-pause 22:00–06:00</text>
<!-- Databricks SQL -->
<rect x="872" y="200" width="240" height="88" rx="8" fill="#E3F2FD" stroke="#1565C0" stroke-width="1.4" filter="url(#sh)"/>
<rect x="872" y="200" width="240" height="30" rx="8" fill="#1565C0"/>
<rect x="872" y="218" width="240" height="12" fill="#1565C0"/>
<text x="992" y="220" text-anchor="middle" font-size="13" font-weight="bold" fill="white">6b Databricks SQL</text>
<text x="992" y="248" text-anchor="middle" font-size="11" font-weight="bold" fill="#0D47A1">SQL Warehouses (Serverless)</text>
<text x="992" y="266" text-anchor="middle" font-size="10" fill="#555">JDBC/ODBC · Ad-hoc queries</text>
<text x="992" y="282" text-anchor="middle" font-size="9" fill="#888">snet-sqlwarehouse-prod (10.10.9.0/24)</text>
<path d="M 765 755 L 795 755 Q 810 755 810 740 L 810 260 Q 810 248 830 248 L 870 248" stroke="#1565C0" stroke-width="1.5" fill="none" marker-end="url(#ar)"/>
<!-- ML / AI -->
<rect x="872" y="304" width="240" height="88" rx="8" fill="#E8F5E9" stroke="#2E7D32" stroke-width="1.4" filter="url(#sh)"/>
<rect x="872" y="304" width="240" height="30" rx="8" fill="#2E7D32"/>
<rect x="872" y="322" width="240" height="12" fill="#2E7D32"/>
<text x="992" y="324" text-anchor="middle" font-size="13" font-weight="bold" fill="white">6c ML / AI</text>
<text x="992" y="352" text-anchor="middle" font-size="11" font-weight="bold" fill="#1B5E20">MLflow · Feature Store · Model Serving</text>
<text x="992" y="370" text-anchor="middle" font-size="10" fill="#555">GPU: NC6s_v3 · dbw-mlops-prod</text>
<text x="992" y="386" text-anchor="middle" font-size="9" fill="#888">Azure OpenAI + RAG (Vector Search)</text>
<path d="M 765 768 L 785 768 Q 800 768 800 752 L 800 355 Q 800 345 820 345 L 870 345" stroke="#2E7D32" stroke-width="1.5" fill="none" marker-end="url(#ar)"/>
<!-- SAS Viya -->
<rect x="872" y="408" width="240" height="105" rx="8" fill="#FFF3E0" stroke="#E65100" stroke-width="1.4" filter="url(#sh)"/>
<rect x="872" y="408" width="240" height="30" rx="8" fill="#E65100"/>
<rect x="872" y="426" width="240" height="12" fill="#E65100"/>
<text x="992" y="428" text-anchor="middle" font-size="13" font-weight="bold" fill="white">6d SAS Viya Compute</text>
<text x="992" y="456" text-anchor="middle" font-size="11" font-weight="bold" fill="#BF360C">AKS: E16s_v5 / E32s_v5</text>
<text x="992" y="474" text-anchor="middle" font-size="10" fill="#555">JDBC LIBNAME → SQL Warehouses</text>
<text x="992" y="490" text-anchor="middle" font-size="10" fill="#555">Actuarial · Risk · Batch scoring</text>
<text x="992" y="506" text-anchor="middle" font-size="9" fill="#E65100">vnet-sas-prod-cc (via hub peering)</text>
<path d="M 765 780 L 775 780 Q 790 780 790 764 L 790 462 Q 790 450 810 450 L 870 450" stroke="#E65100" stroke-width="1.5" fill="none" marker-end="url(#ar)"/>
<!-- SAS write-back path -->
<rect x="872" y="530" width="240" height="56" rx="6" fill="#FFF8E1" stroke="#FFA000" stroke-width="1"/>
<text x="992" y="550" text-anchor="middle" font-size="10" font-weight="bold" fill="#E65100">SAS Write-back Path</text>
<text x="992" y="566" text-anchor="middle" font-size="9" fill="#555">stadlsstagingprod → /staging/sas_writeback/</text>
<text x="992" y="580" text-anchor="middle" font-size="9" fill="#006B3F">Databricks validates + promotes → Gold</text>
<!-- Write-back arrow from SAS to staging to Gold -->
<path d="M 992 586 L 992 604 Q 992 614 980 614 L 690 614 Q 675 614 675 630" stroke="#006B3F" stroke-width="1.2" stroke-dasharray="5,3" fill="none" marker-end="url(#arG)"/>
<!-- Fabric IQ (future, dashed) -->
<rect x="872" y="604" width="240" height="56" rx="8" fill="#F3E5F5" stroke="#7B1FA2" stroke-width="1.3" stroke-dasharray="6,3" filter="url(#sh)"/>
<text x="992" y="624" text-anchor="middle" font-size="11" font-weight="bold" fill="#6A1B9A">Fabric IQ (Horizon 2–3)</text>
<text x="992" y="642" text-anchor="middle" font-size="10" fill="#555">Ontology · Graph · Data/Ops Agents</text>
<text x="992" y="656" text-anchor="middle" font-size="9" fill="#999">Pending GA + UC enforcement validation</text>
<!-- ═══════════════════════════════════════════════════════ -->
<!-- PRIVATE ENDPOINTS & NETWORK (far right column) -->
<!-- ═══════════════════════════════════════════════════════ -->
<rect x="1150" y="80" width="145" height="580" rx="8" fill="none" stroke="#1565C0" stroke-width="1.3" stroke-dasharray="6,3"/>
<rect x="1150" y="80" width="145" height="30" rx="8" fill="#1565C0"/>
<rect x="1150" y="98" width="145" height="12" fill="#1565C0"/>
<text x="1222" y="100" text-anchor="middle" font-size="12" font-weight="bold" fill="white">Network Security</text>
<text x="1222" y="128" text-anchor="middle" font-size="11" font-weight="bold" fill="#1565C0">Private Endpoints</text>
<text x="1222" y="144" text-anchor="middle" font-size="9" fill="#555">snet-private-endpoints</text>
<text x="1222" y="158" text-anchor="middle" font-size="9" fill="#555">10.10.8.0/24</text>
<rect x="1160" y="172" width="125" height="20" rx="3" fill="#E3F2FD"/><text x="1222" y="186" text-anchor="middle" font-size="9" fill="#1565C0">PE: ADLS Bronze</text>
<rect x="1160" y="198" width="125" height="20" rx="3" fill="#E3F2FD"/><text x="1222" y="212" text-anchor="middle" font-size="9" fill="#1565C0">PE: ADLS Silver</text>
<rect x="1160" y="224" width="125" height="20" rx="3" fill="#E3F2FD"/><text x="1222" y="238" text-anchor="middle" font-size="9" fill="#1565C0">PE: ADLS Gold</text>
<rect x="1160" y="250" width="125" height="20" rx="3" fill="#E3F2FD"/><text x="1222" y="264" text-anchor="middle" font-size="9" fill="#1565C0">PE: ADLS Staging</text>
<rect x="1160" y="276" width="125" height="20" rx="3" fill="#E3F2FD"/><text x="1222" y="290" text-anchor="middle" font-size="9" fill="#1565C0">PE: Key Vault ×2</text>
<rect x="1160" y="302" width="125" height="20" rx="3" fill="#E3F2FD"/><text x="1222" y="316" text-anchor="middle" font-size="9" fill="#1565C0">PE: Purview</text>
<rect x="1160" y="328" width="125" height="20" rx="3" fill="#E3F2FD"/><text x="1222" y="342" text-anchor="middle" font-size="9" fill="#1565C0">PE: Databricks</text>
<rect x="1160" y="354" width="125" height="20" rx="3" fill="#E3F2FD"/><text x="1222" y="368" text-anchor="middle" font-size="9" fill="#1565C0">PE: Event Hub</text>
<rect x="1160" y="380" width="125" height="20" rx="3" fill="#E3F2FD"/><text x="1222" y="394" text-anchor="middle" font-size="9" fill="#1565C0">PE: ADF / ACR</text>
<text x="1222" y="424" text-anchor="middle" font-size="10" font-weight="bold" fill="#1565C0">~25–30 PEs total</text>
<text x="1222" y="440" text-anchor="middle" font-size="9" fill="#555">Public access DISABLED</text>
<!-- NSG -->
<rect x="1160" y="460" width="125" height="46" rx="5" fill="#FFEBEE" stroke="#EF5350" stroke-width="0.8"/>
<text x="1222" y="478" text-anchor="middle" font-size="10" font-weight="bold" fill="#C62828">NSG Rules</text>
<text x="1222" y="494" text-anchor="middle" font-size="9" fill="#555">Default: DENY ALL</text>
<text x="1222" y="504" text-anchor="middle" font-size="9" fill="#555">Explicit allows only</text>
<!-- Firewall -->
<rect x="1160" y="516" width="125" height="46" rx="5" fill="#FFF3E0" stroke="#FFA000" stroke-width="0.8"/>
<text x="1222" y="534" text-anchor="middle" font-size="10" font-weight="bold" fill="#E65100">Egress Firewall</text>
<text x="1222" y="550" text-anchor="middle" font-size="9" fill="#555">Azure Firewall Premium</text>
<text x="1222" y="560" text-anchor="middle" font-size="9" fill="#555">FQDN + TLS inspection</text>
<!-- DNS -->
<rect x="1160" y="572" width="125" height="38" rx="5" fill="#E8EAF6" stroke="#5C6BC0" stroke-width="0.8"/>
<text x="1222" y="590" text-anchor="middle" font-size="10" font-weight="bold" fill="#283593">Private DNS</text>
<text x="1222" y="604" text-anchor="middle" font-size="9" fill="#555">Centralized in Hub VNet</text>
<!-- ═══════════════════════════════════════════════════════ -->
<!-- CROSS-CUTTING GOVERNANCE BAR (bottom) -->
<!-- ═══════════════════════════════════════════════════════ -->
<rect x="30" y="690" width="162" height="148" rx="8" fill="#F5F5F5" stroke="#999" stroke-width="1" stroke-dasharray="4,3"/>
<text x="111" y="712" text-anchor="middle" font-size="10" font-weight="bold" fill="#555">DR: Canada East</text>
<text x="111" y="730" text-anchor="middle" font-size="9" fill="#777">GRS / RA-GRS</text>
<text x="111" y="746" text-anchor="middle" font-size="9" fill="#777">Tier 1: RPO≤1h, RTO≤4h</text>
<text x="111" y="762" text-anchor="middle" font-size="9" fill="#777">IaC rebuild</text>
<text x="111" y="778" text-anchor="middle" font-size="9" fill="#777">(Terraform)</text>
<text x="111" y="800" text-anchor="middle" font-size="9" fill="#AAA">Canada East</text>
<text x="111" y="816" text-anchor="middle" font-size="9" fill="#AAA">Warm standby VNets</text>
<text x="111" y="832" text-anchor="middle" font-size="9" fill="#AAA">ExpressRoute secondary</text>
<rect x="195" y="850" width="1100" height="42" rx="6" fill="url(#gTeal)" filter="url(#sh)"/>
<text x="745" y="870" text-anchor="middle" font-size="14" font-weight="bold" fill="white">Cross-Cutting: Governance · Security · Observability</text>
<!-- Tier boxes inside governance bar -->
<rect x="210" y="878" width="188" height="0" rx="0" fill="none"/>
<!-- Sub-labels under governance bar -->
<rect x="1316" y="80" width="244" height="760" rx="8" fill="none" stroke="none"/>
<!-- LEGEND -->
<rect x="1320" y="690" width="240" height="152" rx="8" fill="white" stroke="#DDD" stroke-width="1" filter="url(#sh)"/>
<text x="1440" y="712" text-anchor="middle" font-size="12" font-weight="bold" fill="#333">Legend</text>
<rect x="1332" y="722" width="16" height="12" rx="2" fill="url(#gBronze)" stroke="#B8860B" stroke-width="0.5"/>
<text x="1356" y="732" font-size="10" fill="#555">Bronze (Raw)</text>
<rect x="1440" y="722" width="16" height="12" rx="2" fill="url(#gSilver)" stroke="#78909C" stroke-width="0.5"/>
<text x="1464" y="732" font-size="10" fill="#555">Silver (Curated)</text>
<rect x="1332" y="742" width="16" height="12" rx="2" fill="url(#gGold)" stroke="#C6A300" stroke-width="0.5"/>
<text x="1356" y="752" font-size="10" fill="#555">Gold (Products)</text>
<rect x="1440" y="742" width="16" height="12" rx="2" fill="url(#gGreen)"/>
<text x="1464" y="752" font-size="10" fill="#555">Databricks</text>
<line x1="1332" y1="770" x2="1366" y2="770" stroke="#444" stroke-width="2" marker-end="url(#ar)"/>
<text x="1378" y="774" font-size="10" fill="#555">Data flow</text>
<line x1="1440" y1="770" x2="1474" y2="770" stroke="#444" stroke-width="1.2" stroke-dasharray="5,3"/>
<text x="1486" y="774" font-size="10" fill="#555">Write-back</text>
<rect x="1332" y="788" width="16" height="12" rx="2" fill="none" stroke="#1565C0" stroke-width="1" stroke-dasharray="3,2"/>
<text x="1356" y="798" font-size="10" fill="#555">Network boundary</text>
<rect x="1440" y="788" width="16" height="12" rx="2" fill="none" stroke="#7B1FA2" stroke-width="1" stroke-dasharray="4,2"/>
<text x="1464" y="798" font-size="10" fill="#555">Future (Horizon 2–3)</text>
<rect x="1332" y="812" width="16" height="12" rx="2" fill="url(#gTeal)"/>
<text x="1356" y="822" font-size="10" fill="#555">Governance</text>
<rect x="1440" y="812" width="16" height="12" rx="2" fill="#FFF8E1" stroke="#FFA000" stroke-width="0.5"/>
<text x="1464" y="822" font-size="10" fill="#555">Quality gate</text>
<!-- Governance tier labels below bar -->
<text x="280" y="894" text-anchor="middle" font-size="10" fill="white" font-weight="bold">Tier 1: Microsoft Purview</text>
<text x="280" y="888" text-anchor="middle" font-size="8" fill="#B2DFDB">Glossary · Classification · DLP · DQ</text>
<text x="530" y="894" text-anchor="middle" font-size="10" fill="white" font-weight="bold">Tier 2: Unity Catalog</text>
<text x="530" y="888" text-anchor="middle" font-size="8" fill="#B2DFDB">RLS · CLS · DDM · Column lineage</text>
<text x="760" y="894" text-anchor="middle" font-size="10" fill="white" font-weight="bold">Tier 3: Manta (IBM)</text>
<text x="760" y="888" text-anchor="middle" font-size="8" fill="#B2DFDB">Cross-platform code-level lineage</text>
<text x="970" y="894" text-anchor="middle" font-size="10" fill="white" font-weight="bold">Key Vault (CMK)</text>
<text x="970" y="888" text-anchor="middle" font-size="8" fill="#B2DFDB">Secrets · Encryption · PE-only</text>
<text x="1180" y="894" text-anchor="middle" font-size="10" fill="white" font-weight="bold">Monitor + Sentinel</text>
<text x="1180" y="888" text-anchor="middle" font-size="8" fill="#B2DFDB">Log Analytics · SIEM · Alerts</text>
<!-- VNet boundary label -->
<text x="1128" y="780" font-size="9" fill="#2874A6" font-weight="bold">vnet-data-prod-cc</text>
<text x="1128" y="794" font-size="9" fill="#2874A6">10.10.0.0/16</text>
<text x="1128" y="812" font-size="8" fill="#666">Canada Central</text>
</svg>
Reference in New Issue View Git Blame Copy Permalink