# Storage

Source: https://docs.ops0.com/docs/kubernetes/storage

Manage Kubernetes storage resources including PersistentVolumes, PersistentVolumeClaims, StorageClasses, and ResourceQuotas for persistent data storage.

## Storage Resources

PersistentVolumes (PV)

Cluster-wide storage resources provisioned by admin

PersistentVolumeClaims (PVC)

User requests for storage with specific size and access mode

StorageClasses

Storage provisioning templates for dynamic volume creation

ResourceQuotas

Namespace-level storage consumption limits

---

## PersistentVolumes (PV)

Cluster-wide storage resources that exist independently of pods and persist beyond pod lifecycle.

### Viewing PersistentVolumes

| Column | Description |
|--------|-------------|
| **Name** | PV name |
| **Status** | Available, Bound, Released, Failed |
| **Claim** | PVC using this PV (if bound) |
| **Storage Class** | StorageClass used for provisioning |
| **Capacity** | Storage size (e.g., 100Gi) |
| **Access Modes** | ReadWriteOnce, ReadOnlyMany, ReadWriteMany |
| **Reclaim Policy** | Retain, Delete, or Recycle |
| **Age** | Time since creation |

### PV Status Lifecycle

| Status | Meaning |
|--------|---------|
| **Available** | PV is free and not yet bound to a PVC |
| **Bound** | PV is bound to a PVC and in use |
| **Released** | PVC was deleted but PV not yet reclaimed |
| **Failed** | Automatic reclamation failed |

### PV Details

Click a PV to view:

- **Capacity**: Storage size in Gi
- **Access Modes**:
  - `ReadWriteOnce` (RWO) - Single node read/write
  - `ReadOnlyMany` (ROX) - Multiple nodes read-only
  - `ReadWriteMany` (RWX) - Multiple nodes read/write
- **Reclaim Policy**: What happens when PVC is deleted
- **Storage Class**: Provisioner used to create PV
- **Volume Source**: AWS EBS, GCE PD, NFS, etc.
- **Node Affinity**: Node constraints for volume attachment

### Reclaim Policies

| Policy | Behavior |
|--------|----------|
| **Retain** | PV remains after PVC deletion (manual cleanup needed) |
| **Delete** | PV and underlying storage deleted automatically |
| **Recycle** | Data scrubbed, PV becomes available again (deprecated) |

---

## PersistentVolumeClaims (PVC)

User requests for storage that bind to PersistentVolumes or trigger dynamic provisioning.

### Viewing PersistentVolumeClaims

| Column | Description |
|--------|-------------|
| **Name** | PVC name |
| **Namespace** | Kubernetes namespace |
| **Status** | Pending, Bound, Lost |
| **Volume** | Bound PV name |
| **Capacity** | Requested storage size |
| **Access Modes** | RWO, ROX, RWX |
| **Storage Class** | StorageClass for dynamic provisioning |
| **Age** | Time since creation |

### PVC Status

| Status | Meaning | Action Needed |
|--------|---------|---------------|
| **Bound** | PVC bound to PV, ready to use | None |
| **Pending** | Waiting for PV or provisioning | Check StorageClass, capacity, node availability |
| **Lost** | Bound PV deleted or unavailable | Recreate PV or restore volume |

### PVC Binding Process

### PVC Created
User creates PVC requesting specific capacity and access mode.

### Matching PV Search
Kubernetes looks for available PV matching requirements.

### Bind or Provision
- **Static Provisioning**: Bind to existing PV
- **Dynamic Provisioning**: Create new PV using StorageClass

### Bound
PVC status changes to Bound, ready for pod to use.

### Using PVCs in Pods

PVCs are mounted into pods to provide persistent storage:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    volumeMounts:
    - name: html
      mountPath: /usr/share/nginx/html
  volumes:
  - name: html
    persistentVolumeClaim:
      claimName: nginx-pvc
```

### Expanding PVCs

Some StorageClasses allow volume expansion:

### Check StorageClass
Verify `allowVolumeExpansion: true` is set.

### Edit PVC
Increase the `spec.resources.requests.storage` value.

### Expansion Triggers
Kubernetes expands the underlying volume.

### Wait for Completion
PVC status shows `FileSystemResizePending` until complete.

### Verify New Size
Check pod sees expanded storage capacity.

**Expansion Limitations**

- Volume expansion is one-way only (cannot shrink)
- Some volume types require pod restart to recognize new size
- Not all StorageClasses support expansion

---

## StorageClasses

Templates for dynamic PersistentVolume provisioning with cloud provider-specific parameters.

### Viewing StorageClasses

| Column | Description |
|--------|-------------|
| **Name** | StorageClass name |
| **Provisioner** | Volume plugin (ebs.csi.aws.com, pd.csi.storage.gke.io, etc.) |
| **Reclaim Policy** | Delete or Retain |
| **Volume Binding Mode** | Immediate or WaitForFirstConsumer |
| **Allow Expansion** | Whether volumes can be resized |
| **Default** | Is this the default StorageClass |

### Common StorageClasses by Provider

#### AWS EKS

| Name | Provisioner | Type | IOPS | Use Case |
|------|-------------|------|------|----------|
| **gp3** | ebs.csi.aws.com | General Purpose SSD | 3000-16000 | Default, balanced performance |
| **gp2** | ebs.csi.aws.com | General Purpose SSD | Burstable | Legacy default |
| **io2** | ebs.csi.aws.com | Provisioned IOPS SSD | 100-64000 | High performance databases |
| **st1** | ebs.csi.aws.com | Throughput Optimized HDD | - | Big data, log processing |

#### Google GKE

| Name | Provisioner | Type | Use Case |
|------|-------------|------|----------|
| **standard** | pd.csi.storage.gke.io | Standard PD | Default, cost-effective |
| **balanced** | pd.csi.storage.gke.io | Balanced PD | General purpose workloads |
| **ssd** | pd.csi.storage.gke.io | SSD PD | High performance |
| **extreme** | pd.csi.storage.gke.io | Extreme PD | Highest performance |

#### Azure AKS

| Name | Provisioner | Type | Use Case |
|------|-------------|------|----------|
| **default** | disk.csi.azure.com | Standard HDD | Low cost, infrequent access |
| **managed-premium** | disk.csi.azure.com | Premium SSD | Production workloads |
| **managed-csi-premium** | disk.csi.azure.com | Premium SSD | High performance |
| **azurefile** | file.csi.azure.com | Azure Files | ReadWriteMany support |

### Volume Binding Modes

| Mode | Behavior | Use When |
|------|----------|----------|
| **Immediate** | PV provisioned immediately when PVC created | Single-zone clusters, default behavior |
| **WaitForFirstConsumer** | PV provisioned when pod using PVC is scheduled | Multi-zone clusters, ensures volume in same zone as pod |

### StorageClass Parameters

**AWS EBS Example:**

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  iops: "5000"
  throughput: "250"
  encrypted: "true"
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
reclaimPolicy: Delete
```

**Google PD Example:**

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ssd
provisioner: pd.csi.storage.gke.io
parameters:
  type: pd-ssd
  replication-type: regional-pd
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
```

### Default StorageClass

Mark a StorageClass as default for PVCs without explicit class:

```yaml
metadata:
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
```

**Behavior:**
- PVCs without `storageClassName` use default class
- Only one default class should exist per cluster
- Helps simplify PVC creation for users

---

## ResourceQuotas

Limit storage consumption per namespace to prevent resource exhaustion.

### Viewing ResourceQuotas

| Column | Description |
|--------|-------------|
| **Name** | Quota name |
| **Namespace** | Kubernetes namespace |
| **Resource** | What is limited (storage, PVCs, etc.) |
| **Used** | Current consumption |
| **Hard** | Maximum allowed |

### Storage Quota Types

| Quota Type | Description |
|------------|-------------|
| **requests.storage** | Total storage requested across all PVCs |
| **persistentvolumeclaims** | Maximum number of PVCs allowed |
| **`<storageclass>`.storageclass.storage.k8s.io/requests.storage** | Storage limit for specific StorageClass |
| **`<storageclass>`.storageclass.storage.k8s.io/persistentvolumeclaims** | PVC count limit for specific StorageClass |

### Example ResourceQuota

```yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: storage-quota
  namespace: development
spec:
  hard:
    requests.storage: "500Gi"
    persistentvolumeclaims: "20"
    gp3.storageclass.storage.k8s.io/requests.storage: "300Gi"
    gp3.storageclass.storage.k8s.io/persistentvolumeclaims: "15"
```

**Enforcement:**
```
Used / Hard
─────────────────────────────────────
Storage (All):          250Gi / 500Gi
PVC Count:              12 / 20
Storage (gp3):          180Gi / 300Gi
PVC Count (gp3):        10 / 15
```

### Quota Exceeded Behavior

When quota is exceeded, PVC creation fails:

```
Error creating PVC:
Exceeded quota: storage-quota
Requested: requests.storage=100Gi
Used: 450Gi
Hard: 500Gi
```

**Resolution:**
- Request quota increase
- Delete unused PVCs
- Use different StorageClass with available quota
- Archive data to cheaper storage

---

## Example: StatefulSet with Persistent Storage

### Scenario

Deploy PostgreSQL database with persistent storage using StatefulSet and PVC.

### StatefulSet Configuration

```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgresql
  namespace: production
spec:
  serviceName: postgresql
  replicas: 3
  selector:
    matchLabels:
      app: postgresql
  template:
    metadata:
      labels:
        app: postgresql
    spec:
      containers:
      - name: postgresql
        image: postgres:15
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgresql-secret
              key: password
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "gp3"
      resources:
        requests:
          storage: 100Gi
```

### Created Resources

**PVCs Created:**
```
NAME                  STATUS   VOLUME                 CAPACITY   STORAGE CLASS
data-postgresql-0     Bound    pvc-abc123             100Gi      gp3
data-postgresql-1     Bound    pvc-def456             100Gi      gp3
data-postgresql-2     Bound    pvc-ghi789             100Gi      gp3
```

**PVs Auto-Provisioned:**
```
NAME         CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM
pvc-abc123   100Gi      RWO            Delete           Bound    production/data-postgresql-0
pvc-def456   100Gi      RWO            Delete           Bound    production/data-postgresql-1
pvc-ghi789   100Gi      RWO            Delete           Bound    production/data-postgresql-2
```

### Scaling Behavior

**Scale to 5 replicas:**
```
kubectl scale statefulset postgresql --replicas=5
```

**New PVCs created automatically:**
```
data-postgresql-3     Pending   -                      -          gp3
data-postgresql-4     Pending   -                      -          gp3
```

**After provisioning:**
```
data-postgresql-3     Bound     pvc-jkl012             100Gi      gp3
data-postgresql-4     Bound     pvc-mno345             100Gi      gp3
```

**Total Storage Used:** 5 × 100Gi = 500Gi

---

## Troubleshooting Storage Issues

PVC Stuck in Pending

**Causes:**

- No PV matches requested capacity/access mode

- StorageClass not found or misconfigured

- ResourceQuota exceeded

- Volume provisioning failed (check events)

**Solution:** Check PVC events with `kubectl describe pvc [name]`. Verify StorageClass exists and provisioner is healthy.

Pod Stuck: FailedAttachVolume

**Causes:**

- Volume already attached to different node

- Volume doesn't exist in cloud provider

- Node doesn't have permissions to attach volume

- Volume zone doesn't match node zone

**Solution:** Check pod events. Verify PV exists in cloud console. Ensure node and volume are in same availability zone.

PVC Won't Delete

**Causes:**

- PVC is still in use by a pod (protection finalizer)

- PVC has deletion finalizer preventing removal

**Solution:** Delete or scale down pods using the PVC first. Check for finalizers with `kubectl get pvc [name] -o yaml`. Never force-remove finalizers without understanding implications.

Volume Expansion Failed

**Causes:**

- StorageClass doesn't allow expansion

- Requested size exceeds cloud provider limits

- Volume type doesn't support online resizing

**Solution:** Verify `allowVolumeExpansion: true` in StorageClass. Some volumes require pod restart after expansion. Check cloud provider volume limits.

Best Practices

• **Use WaitForFirstConsumer** - Ensures volume provisioned in same zone as pod (multi-zone clusters)

• **Set Resource Quotas** - Prevent runaway storage costs with namespace quotas

• **Enable Volume Expansion** - Allow resizing without recreating PVCs

• **Choose Appropriate StorageClass** - Match performance requirements (gp3 vs io2) to workload needs

• **Monitor PV Usage** - Set up alerts for storage capacity nearing limits

• **Backup Critical Data** - PVs can fail; use volume snapshots or backups for databases