Network Isolation

Overview

QHx Manager automatically generates Kubernetes NetworkPolicies based on MLS labels applied by the Admission Controller. This enforces network-level isolation between workloads with different classification levels, compartments, or releasabilities.

Workloads with identical MLS labels can communicate freely. Workloads with different MLS labels are isolated by default.

How It Works

The network isolation controller continuously monitors Kubernetes resources:

1. Identify MLS Nodes: Group resources by their MLS label combination
2. Generate Policies: Create NetworkPolicy resources for each MLS node
3. Enforce Isolation: Default-deny traffic between different MLS nodes
4. Allow Internal Traffic: Permit traffic within the same MLS node

All NetworkPolicy creation and updates happen automatically. No manual configuration required.

MLS Identity and Nodes

MLS Identity

An MLS identity is the canonical representation of a resource’s MLS labels.

Example:

Resource with labels:

metadata:
  labels:
    mls.qhx.dev/level: "us:s"
    mls.qhx.dev/compartment: "quantum"
    mls.qhx.dev/releasability: "us,uk"

MLS identity (conceptual notation): us:s(compartment=quantum, releasability=us,uk)

MLS Node

An MLS node is the set of all Kubernetes resources sharing the same MLS identity.

Example:

These pods belong to the same MLS node:

# Pod 1
metadata:
  labels:
    mls.qhx.dev/level: "us:s"
    mls.qhx.dev/compartment: "quantum"

# Pod 2
metadata:
  labels:
    mls.qhx.dev/level: "us:s"
    mls.qhx.dev/compartment: "quantum"

These pods belong to different MLS nodes:

# Pod 3
metadata:
  labels:
    mls.qhx.dev/level: "us:s"
    mls.qhx.dev/compartment: "quantum"

# Pod 4
metadata:
  labels:
    mls.qhx.dev/level: "us:s"
    mls.qhx.dev/compartment: "tempest"  # Different compartment

Default Isolation Model

Each MLS node is isolated by default:

• Ingress: Only pods within the same MLS node can send traffic
• Egress: Only pods within the same MLS node can receive traffic
• External traffic: Blocked by default (see Exempted Flows below)

Generated NetworkPolicy

For an MLS node with identity us:s(compartment=quantum):

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: qhx-mls-us-s-quantum
  namespace: default
spec:
  podSelector:
    matchLabels:
      mls.qhx.dev/level: "us:s"
      mls.qhx.dev/compartment: "quantum"
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          mls.qhx.dev/level: "us:s"
          mls.qhx.dev/compartment: "quantum"
  egress:
  - to:
    - podSelector:
        matchLabels:
          mls.qhx.dev/level: "us:s"
          mls.qhx.dev/compartment: "quantum"

This policy:

• Applies to all pods with matching MLS labels
• Allows ingress only from pods with identical labels
• Allows egress only to pods with identical labels
• Implicitly denies all other traffic

Pod Selector Binding

NetworkPolicies bind to pods using label selectors that match the MLS identity exactly:

podSelector:
  matchLabels:
    mls.qhx.dev/level: "us:s"
    mls.qhx.dev/compartment: "quantum"
    mls.qhx.dev/releasability: "us,uk"

This approach scales efficiently because:

• No need to list individual pod IPs
• Policies update automatically as pods are created/destroyed
• Single NetworkPolicy covers all pods in an MLS node

Exempted Flows

The default isolation model blocks external connections and cross-MLS-node communication. Several options enable controlled exceptions.

External Connections

Allow traffic to/from outside the cluster:

apiVersion: qhx.dev/v1
kind: QHxNetworkPolicy
metadata:
  name: allow-external-ingress
  namespace: production
spec:
  mlsIdentity:
    level: "us:s"
    compartment: "quantum"
  externalIngress:
    enabled: true
    sources:
      - 203.0.113.0/24  # External load balancer
  externalEgress:
    enabled: true
    destinations:
      - 198.51.100.0/24  # External database

Use cases:

• Allow ingress from external load balancers
• Allow egress to external databases or APIs
• Enable connections to devices outside Kubernetes

Cross-Node L4 Rules

Allow specific traffic between different MLS nodes:

apiVersion: qhx.dev/v1
kind: QHxNetworkPolicy
metadata:
  name: allow-cross-compartment
spec:
  mlsIdentity:
    level: "us:s"
    compartment: "quantum"
  allowFrom:
    - level: "us:s"
      compartment: "tempest"
  allowTo:
    - level: "us:s"
      compartment: "tempest"

Warning: Cross-node rules bypass MLS isolation. Use sparingly and only when required.

DNS and System Services

Common exemptions for cluster infrastructure:

apiVersion: qhx.dev/v1
kind: QHxNetworkPolicy
metadata:
  name: system-services
spec:
  mlsIdentity:
    level: "us:s"
    compartment: "quantum"
  allowEgress:
    - toNamespace: kube-system
      toPods:
        matchLabels:
          k8s-app: kube-dns
    - toCIDR: 169.254.169.254/32  # AWS metadata service

CNI Integration

Standard Kubernetes NetworkPolicy

QHx uses standard Kubernetes NetworkPolicy resources. Compatible with:

• Calico
• Cilium
• Weave Net
• Any CNI implementing NetworkPolicy

No CNI-specific extensions required for basic functionality.

Cilium Extensions

Cilium provides additional features for scalability:

CiliumCIDRGroup:

Reference CIDR lists indirectly instead of embedding IPs:

apiVersion: cilium.io/v2alpha1
kind: CiliumCIDRGroup
metadata:
  name: external-databases
spec:
  externalCIDRs:
    - 198.51.100.10/32
    - 198.51.100.11/32
---
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: quantum-egress
spec:
  endpointSelector:
    matchLabels:
      mls.qhx.dev/level: "us:s"
      mls.qhx.dev/compartment: "quantum"
  egress:
  - toCIDRSet:
    - cidrGroupRef: external-databases

This improves performance when CIDR lists change frequently.

Scalability

Pod Selector Efficiency

Label-based selectors scale efficiently:

• Not scalable: Listing every pod IP explicitly
• Scalable: Using label selectors (QHx approach)

QHx creates one NetworkPolicy per MLS node, regardless of pod count.

Example:

• 100 pods in MLS node us:s(quantum) → 1 NetworkPolicy
• 1000 pods added → Same NetworkPolicy, no updates needed

Foreign Service Handling

Services outside the cluster must be referenced by IP:

egress:
- to:
  - ipBlock:
      cidr: 198.51.100.10/32

Scalability considerations:

• Static external services: No scalability issues
• Dynamic external services: May cause NetworkPolicy churn
• Use Cilium CiliumCIDRGroup for dynamic services

Monitoring Policy Count

Check number of NetworkPolicies:

kubectl get networkpolicies --all-namespaces | wc -l

One NetworkPolicy per MLS node per namespace is typical.

Troubleshooting

Connectivity Issues

Symptom: Pods cannot communicate despite being in the same MLS node.

Check pod labels match exactly:

# Check pod 1 labels
kubectl get pod pod-1 -o jsonpath='{.metadata.labels}' | jq

# Check pod 2 labels
kubectl get pod pod-2 -o jsonpath='{.metadata.labels}' | jq

Common issues:

• Missing MLS label on one pod
• Different releasability values
• Typo in compartment label

NetworkPolicy Not Created

Symptom: No NetworkPolicy exists for an MLS node.

Check QHx Manager logs:

kubectl logs -n qhx-system -l app=qhx-manager | grep "network-policy"

Common issues:

• QHx Manager not running
• NetworkPolicy controller disabled
• No pods exist with the MLS identity yet

External Connections Blocked

Symptom: Cannot reach external services.

Check if external egress is configured:

kubectl get qhxnetworkpolicy -A

Verify NetworkPolicy allows external traffic:

kubectl get networkpolicy <policy-name> -o yaml

Add exemption for external destinations:

apiVersion: qhx.dev/v1
kind: QHxNetworkPolicy
metadata:
  name: allow-external-db
spec:
  mlsIdentity:
    level: "us:s"
    compartment: "quantum"
  externalEgress:
    enabled: true
    destinations:
      - 198.51.100.0/24

CNI Not Enforcing Policies

Verify CNI supports NetworkPolicy:

# Check CNI pods are running
kubectl get pods -n kube-system | grep -E 'calico|cilium|weave'

# Check NetworkPolicy CRD exists
kubectl get crd networkpolicies.networking.k8s.io

Test with simple NetworkPolicy:

kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-deny-all
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
EOF

# Try to connect to pod - should fail
kubectl exec -it test-pod -- curl http://other-pod

Verify Policy Application

Check which NetworkPolicies apply to a pod:

POD_LABELS=$(kubectl get pod my-pod -o json | jq -r '.metadata.labels')
kubectl get networkpolicies -o json | \
  jq --argjson labels "$POD_LABELS" \
  '.items[] | select(.spec.podSelector.matchLabels | . as $sel | $labels | contains($sel))'

Integration with QHx Features

With Admission Controller

Network isolation relies on consistent MLS labeling:

1. Admission Controller applies MLS labels to pods
2. Network isolation controller detects labeled pods
3. NetworkPolicies are generated based on labels
4. CNI enforces policies

Without the Admission Controller, manual MLS labeling is required for network isolation to function.

With QHx Proxy

Network isolation and QHx Proxy provide complementary security:

• Network isolation: L3/L4 (IP/port) enforcement
• QHx Proxy: L7 (application protocol) enforcement with workload identity

Combined defense:

1. NetworkPolicy prevents unauthorized IP connections
2. QHx Proxy validates SPIFFE identity even if IP connection succeeds
3. Application receives request only if both checks pass

With Flowspecs

Flowspecs configure QHx Proxy but do not affect NetworkPolicies:

# This configures proxy sidecars
qhx.dev/flows: |
  http from app=client

NetworkPolicies are generated based only on MLS labels, not flowspecs.

Ensure client and server pods have compatible MLS labels if using flowspecs.

Security Considerations

Defense in Depth

Network isolation provides one layer of security:

• Layer 1: NetworkPolicy (IP-level isolation)
• Layer 2: QHx Proxy (workload identity verification)
• Layer 3: Application authorization (business logic)

Do not rely solely on NetworkPolicy for security.

Label Tampering

NetworkPolicies enforce isolation based on MLS labels. If labels can be modified:

• Attacker could change pod labels to join different MLS node
• Attacker could remove labels to bypass policies

Mitigations:

• Use Admission Controller to prevent label modification
• Seal QHxClusterPolicy to prevent policy changes
• Restrict RBAC permissions on pod label updates

Exemption Auditing

Cross-MLS-node exemptions bypass isolation controls. Track and audit all exemptions:

# List all QHx NetworkPolicy exemptions
kubectl get qhxnetworkpolicy -A -o yaml | grep -A 10 "allowFrom\|allowTo"

Review exemptions quarterly and remove unnecessary rules.

DNS Leakage

NetworkPolicies may not block DNS queries depending on CNI implementation:

# Explicitly allow DNS egress
egress:
- to:
  - namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: kube-system
  - podSelector:
      matchLabels:
        k8s-app: kube-dns
  ports:
  - protocol: UDP
    port: 53

Be aware DNS queries may reveal information about classified services.

Example Configurations

Simple Isolation

Two compartments, no cross-compartment traffic:

# Compartment Quantum
mls.qhx.dev/level: "us:s"
mls.qhx.dev/compartment: "quantum"

# Compartment Tempest
mls.qhx.dev/level: "us:s"
mls.qhx.dev/compartment: "tempest"

Result: Complete isolation between quantum and tempest pods.

Multi-Level with External Access

Secret workloads with external database access:

apiVersion: qhx.dev/v1
kind: QHxNetworkPolicy
metadata:
  name: secret-quantum-external-db
spec:
  mlsIdentity:
    level: "us:s"
    compartment: "quantum"
  externalEgress:
    enabled: true
    destinations:
      - 198.51.100.50/32  # Database server

Cross-Compartment Data Flow

Allow quantum workloads to query tempest services:

apiVersion: qhx.dev/v1
kind: QHxNetworkPolicy
metadata:
  name: quantum-to-tempest
spec:
  mlsIdentity:
    level: "us:s"
    compartment: "quantum"
  allowEgress:
    - level: "us:s"
      compartment: "tempest"
      protocol: TCP
      port: 8080

Warning: This creates a potential data flow path from quantum to tempest. Ensure application-level controls prevent unauthorized data access.

Monitoring

View Generated Policies

# List all NetworkPolicies
kubectl get networkpolicies -A

# View specific policy
kubectl get networkpolicy qhx-mls-us-s-quantum -o yaml

# Count policies per namespace
kubectl get networkpolicies -A --no-headers | awk '{print $1}' | sort | uniq -c

Audit Policy Changes

Monitor NetworkPolicy events:

kubectl get events -A --field-selector involvedObject.kind=NetworkPolicy

Test Connectivity

Verify isolation between MLS nodes:

# From quantum pod
kubectl exec -it quantum-pod -- curl http://tempest-pod:8080
# Should fail

# From quantum pod to another quantum pod
kubectl exec -it quantum-pod -- curl http://quantum-pod-2:8080
# Should succeed

Related Documentation

• Admission Controller
• QHx MLS Label Format Definition
• Cluster Policy Reference
• QHx Architecture