Kubernetes Platform Landing Zone & Fleet Infrastructure
To implement the 5-Tier Medallion architecture, a robust Kubernetes platform is required. The platform must support GPU-accelerated workloads, strict HIPAA compliance, and GitOps fleet management.
Management plane
ns: rancher-system · argocd · crossplane-system · backstage · vault-system
Rancher Prime (MCM)Cluster API (CAPI)ArgoCD + ApplicationSetsFlux CDCrossplaneBackstage IDPHashiCorp VaultExternal Secrets OperatorSOPScert-manager (root CA)
Cluster provisioning — 6 flavors
All clusters bootstrapped and day-2 configured from management plane via GitOps ApplicationSets
Cloud managed
EKS (AWS)AKS (Azure)GKE (Google Cloud)
On-premise / self-managed
kubeadmRKE2 (Rancher)OpenShift
Per-cluster platform add-ons
ns: platform-system · monitoring · logging · security · velero
Networking
Cilium CNI (eBPF)Istio service meshGateway APIMetalLB (on-prem)NGINX Ingress
Storage
EBS / Azure Disk / PD CSILonghorn (on-prem HA)Rook-CephVelero backups
Policy & security
OPA GatekeeperKyvernoFalco (runtime)Trivy Operatorcert-managerCosign + SigstoreHarbor Registry
Observability
Prometheus OperatorGrafanaLoki + PromtailJaeger / TempoOpenTelemetry CollectorDCGM ExporterAudit → SIEM
Nvidia GPU & AI infrastructure
ns: gpu-operator · nvidia-network-operator · ai-platform
Nvidia operators
GPU OperatorNetwork OperatorDCGM + DCGM ExporterNode Feature DiscoveryMIG ManagerGPU time-slicing
CUDA stack
CUDA 12.xcuDNN 9NCCL (multi-GPU)TensorRT 10Nvidia Container Toolkit
Inference & serving
Nvidia NIM microservicesBioNeMo NIM (clinical)Triton Inference ServervLLMKServe InferenceService
Nvidia NemoClaw — agentic AI framework
ns: nemo-agents · milvus · redis
NeMo framework
NeMo Core (Megatron-LM)NeMo Customizer (LoRA)NeMo RAFTNeMo EvaluatorNeMo GuardRails
NemoClaw orchestration
Agent SupervisorReAct reasoning engineTool registryMilvus vector DB (RAG)Redis (conversation memory)Multi-agent workflow engineClinical safety guardrails
Healthcare clinical AI agents & data
ns: clinical-apps (PHI) · clinical-data (PHI, deny-all network policy default)
Clinical AI agents (NemoClaw-orchestrated)
Diagnostic decision supportAmbient clinical documentationDrug interaction checkerRadiology AI (DICOM)Prior authorization automationPatient risk stratificationClinical coding (ICD-10 / CPT)Care plan generator
Healthcare data integration
FHIR R4 (Hapi FHIR)HL7 v2/v3 brokerDICOM server (Orthanc)SMART on FHIR (Epic / Cerner)De-identification pipelineClinical data lake
Namespace strategy
| Namespace | Contents | PHI |
|---|---|---|
| kube-system | API server, CoreDNS, kube-proxy, cloud-controller-manager | — |
| platform-system | Istio, cert-manager, ingress, MetalLB, external-dns | — |
| monitoring | Prometheus, Grafana, Loki, Jaeger, OTel Collector | — |
| security | Falco, OPA Gatekeeper, Kyverno, Vault Agent injector | — |
| gpu-operator | Driver daemonset, CUDA, device plugin, DCGM, MIG manager | — |
| ai-platform | NIM, Triton, vLLM, KServe, BioNeMo containers | — |
| nemo-agents | NemoClaw supervisor, tool registry, Milvus, Redis | — |
| clinical-apps | Clinical AI agent workloads (restricted PSS) | ✓ |
| clinical-data | FHIR, HL7, DICOM, de-ID pipeline (deny-all NetworkPolicy) | ✓✓ |
Cloud managed Kubernetes
Cloud — AWS
Amazon EKS
NetworkingAWS VPC CNI · Cilium (overlay option)
StorageEBS CSI (gp3) · EFS CSI (RWX) · S3 Mountpoint
GPU nodesp5.48xlarge (H100) · p4d (A100) · g5 (A10G)
IAMIRSA — IAM roles for service accounts
AutoscalerKarpenter NodePool CRDs
Load balancerAWS Load Balancer Controller (ALB/NLB)
GitOpsEKS Blueprints (Terraform) + ArgoCD
NotableEKS Anywhere bridges to on-prem
Cloud — Azure
Azure AKS
NetworkingAzure CNI Overlay · Cilium BYOC
StorageAzure Disk v2 (Premium SSD) · Azure NetApp Files
GPU nodesNC A100 v4 · ND H100 v5 · NCas T4 v3
IAMWorkload Identity (UAMI + federated creds)
AutoscalerCluster Autoscaler + KEDA
Load balancerAzure Load Balancer + AGIC
GitOpsFlux (AKS GitOps extension) or ArgoCD
NotableAKS Hybrid · confidential compute nodes
Cloud — GCP
Google GKE
NetworkingGKE Dataplane V2 (Cilium eBPF) · VPC-native
StoragePersistent Disk CSI · Hyperdisk ML · Filestore
GPU nodesH100 (A3 mega VMs) · A100 (A2 VMs) · L4 (G2 VMs)
IAMWorkload Identity Federation
AutoscalerCluster Autoscaler + NAP (node auto-provisioning)
Load balancerGKE native Gateway API (L7)
GitOpsConfig Sync + Config Connector
NotableAutopilot mode · GKE Enterprise fleet
On-premise / self-managed Kubernetes
On-premise — bare metal / VM
kubeadm
NetworkingCilium CNI (eBPF, recommended) / Calico
StorageLonghorn · Rook-Ceph · NFS subdir provisioner
GPU nodesGPU Operator manages drivers + device plugin
IAMVault + External Secrets + Keycloak OIDC (Dex)
Load balancerMetalLB (L2 or BGP mode)
HA controlKeepalived + HAProxy for API server VIP
AutoscalerCluster Autoscaler (Cluster API provider)
NotableMaximum flexibility · highest operational burden
On-premise — Rancher
RKE2
NetworkingCanal (default) · Cilium · Calico selectable
StorageLonghorn (Rancher-native, 3-replica HA)
GPU nodesGPU Operator + RKE2 node taints / labels
IAMRancher RBAC · Keycloak / AD + Vault
Load balancerMetalLB + Nginx Ingress (bundled)
HA controletcd HA built-in · Rancher Prime MCM
AutoscalerCluster Autoscaler via Rancher machine pools
NotableFIPS 140-2 mode · CIS hardening built-in
On-premise — Red Hat
OpenShift
NetworkingOVN-Kubernetes (default) · Multus CNI
StorageOpenShift Data Foundation (Ceph / NooBaa)
GPU nodesGPU Operator via OperatorHub + NFD
IAMOAuth + SCC · Keycloak / LDAP / Active Directory
Load balancerHAProxy router (built-in) + MetalLB
HA control3-node etcd HA · multi-AZ capable
AutoscalerMachineSet + MachineAutoscaler CRDs
NotableSCC stricter than K8s PSS · OperatorHub · FIPS
Common add-ons bootstrapped to all 6 flavors via ArgoCD ApplicationSet — cert-manager · Falco · OPA Gatekeeper · Kyverno · Prometheus Operator · Grafana · Loki · Jaeger · OTel Collector · Velero · Harbor · Trivy Operator · Cosign · Nvidia GPU Operator · External Secrets Operator
Hardware
GPU compute nodes
H100 SXM5 80GBA100 SXM4 80GBNVLink 4th genInfiniBand HDR 200Gb/sGPUDirect RDMAnvidia.com/gpu taint on node
Nvidia operators
GPU Operator + Network Operator
GPU Operator (v24.x)Driver daemonsetNvidia Device PluginDCGM + DCGM ExporterNode Feature DiscoveryMIG Manager (A100/H100)GPU time-slicingNetwork Operator (RDMA SR-IOV)
CUDA stack
CUDA runtime and compute libraries
CUDA 12.xcuDNN 9NCCL 2.x (multi-GPU comms)TensorRT 10cuBLASNvidia Container Toolkit
Inference infra
Model serving infrastructure
Nvidia NIM (API-compatible containers)BioNeMo NIM (clinical models)Triton Inference ServervLLM (high-throughput LLM)KServe InferenceService CRDServingRuntime CRDKnative (scale-to-zero)
NeMo framework
Foundation model development and fine-tuning
NeMo Core (Megatron-LM)NeMo Customizer (LoRA / P-Tuning)NeMo RAFT (retrieval fine-tuning)NeMo EvaluatorNeMo GuardRailsPyTorch FSDP / DDP
NemoClaw agents
Agentic AI orchestration layer
NemoClaw Agent SupervisorReAct reasoning engineTool registryMilvus vector DB (semantic memory)Redis (conversation memory)RAG pipelineMulti-agent workflow engineClinical safety guardrails
Clinical tools
Tool registry — registered tools available to NemoClaw agents
FHIR R4 API (patient data)DICOM WADO-RS (imaging)HL7 v2 message brokerDrugBank / OpenFDA APIClinical KB (UpToDate)ICD-10 / CPT encoderSMART on FHIR (Epic / Cerner)Lab reference ranges
Clinical agents
Healthcare AI agents deployed on NemoClaw
Diagnostic decision supportAmbient clinical documentationDrug interaction checkerRadiology report generationPrior authorization automationPatient risk stratificationClinical coding (ICD/CPT)Care plan generator
GPU node sizing guide — 1× H100 80GB for NIM LLM serving (Llama-3.1-70B) · 2–4× A100 for NeMo fine-tuning jobs · 1× A10G for auxiliary agent tasks · MIG mode splits H100 into 7 × 10GB instances for smaller clinical models · DCGM metrics scraped by Prometheus every 30s
Access control & identity
Kubernetes RBAC — least-privilege ClusterRoles + Roles per namespace; no wildcard verbs
OPA Gatekeeper — ConstraintTemplates deny privileged pods, enforce labels, require resource limits in clinical namespaces
Kyverno — restrict image registries to Harbor; block :latest tags in production
Pod Security Standards Restricted profile enforced on clinical-data namespace
HashiCorp Vault dynamic secrets — database credentials TTL 1h; no long-lived secrets in pods
OpenShift SCC — enforces UID ranges, no root, no host path mount for PHI workloads
OIDC SSO + MFA via Keycloak — governs kubectl, Rancher, ArgoCD, and Grafana access
Encryption at rest and in transit
etcd encryption at rest — AES-256-GCM via EncryptionConfiguration; envelope key in cloud KMS / Vault Transit
PVC encryption — KMS-backed StorageClass (gp3/Premium SSD); Longhorn volume encryption on-prem via LUKS
mTLS 100% coverage — Istio PeerAuthentication STRICT in clinical-data and clinical-apps namespaces
Service-to-service JWT validation — Istio AuthorizationPolicy checks SPIFFE identity (SVIDs) on every call
Container image signing — Cosign + Sigstore; Kyverno admission webhook blocks any unsigned image
Secrets in Git encrypted with SOPS + age; External Secrets Operator syncs decrypted values from Vault
Network isolation
Default-deny NetworkPolicy in clinical-data — only explicitly whitelisted ingress/egress permitted
Cilium ClusterwidePolicies — L7 HTTP/gRPC enforcement with Hubble flow visibility per namespace
Istio AuthorizationPolicy — namespace allow-lists; denies traffic from unapproved service accounts
Dedicated GPU node pools with taints — only ai-platform and nemo-agents pods can tolerate and schedule
Private cluster API server — not publicly accessible; access via bastion + VPN + PAM only
Istio egress gateway — all outbound from PHI namespaces is forced through a gateway with audit logging
Audit trail & monitoring
Kubernetes audit log — RequestResponse level for clinical-data; shipped to SIEM (Splunk / Elastic)
Falco runtime rules — alerts on container escape, privilege escalation, PHI file access, anomalous network
Istio access logs — every service call logged with source identity, destination, latency, status
OPA audit mode — all policy violations (deny or warn) logged with resource and user context
HIPAA audit trail — PHI access events tagged with user ID, timestamp, action, resource → immutable log store
DCGM GPU metrics → Prometheus; critical CVE alerts from Trivy Operator page on-call within 15 min
Backup & disaster recovery
Velero backups — cluster state + PV snapshots to encrypted S3/Blob/GCS; schedule every 6h, retain 30 days
etcd snapshots — automated via Rancher / OpenShift; stored in a separate encrypted bucket from application data
FHIR data replication — Hapi FHIR async to secondary region; RPO target < 1h
RTO target: < 4h clinical systems; < 1h agent inference (stateless pods recreated from image)
DICOM archive — dual-site replication with CRC validation; Orthanc on replicated PostgreSQL cluster
Organizational & regulatory
BAA signed with AWS / Azure / GCP for PHI in cloud; BAA addendum with Nvidia for NIM clinical use
Dedicated tenancy — PHI node pools on dedicated bare metal or dedicated hosts (no shared hypervisor)
Just-in-time kubectl access via Vault + OIDC; all privileged sessions recorded (PAM)
SBOM — Syft generates software bill of materials for every production image; stored in Harbor OCI registry
HITRUST CSF mapping — platform capabilities mapped to 80+ controls; evidence collected quarterly
Automated incident runbooks triggered by Falco alerts → PagerDuty escalation + Slack notification