Guides

Day-2 operations

KubeCell is designed to be operated from status, not from tribal knowledge. This page covers the routines that keep a fleet healthy and the actions that are deliberately manual.

Routine health checks

Two questions matter every day: are the Cells alive, and is the inventory fresh enough to make promises? The second is easy to forget and the cause of most "the Plan lied" reports.

health.sh

# Fleet overview: phase, OCM state, lease freshnesskubectl-n kubecell-system get cells \  -o custom-columns=NAME:.metadata.name,PHASE:.status.phase,\JOINED:.status.managedCluster.joined,AVAILABLE:.status.managedCluster.available,\LEASE:.status.managedCluster.leaseFresh,NODES:.status.nodes[*].name# Child clusters and their endpointskubectl-n kubecell-system get vc \  -o custom-columns=NAME:.metadata.name,PHASE:.status.phase,\CELL:.spec.cellRef.name,CLASS:.spec.classRef.name,ENDPOINT:.status.endpoint.address# Is inventory fresh enough to make promises?kubectl-n kubecell-system get cell cell1 \  -o jsonpath='{range .status.conditions[*]}{.type}={.status}{"\n"}{end}'

Lease freshness is the liveness signal

OCM Available=True has been observed to go stale without updates. Treat a fresh managed-cluster-lease as the decisive signal, and InventoryFresh as the precondition for any capacity decision.

Changing a child cluster's size

There is no in-place resize. Vertical scaling is a deliberate move to a different quota tier: publish the new class, create the new child cluster, move workloads, delete the old one.

quota.sh

# 1. Publish a new tier (platform action)kubectl apply-f virtualnodeclass-large.yaml# 2. Create a new child cluster on the larger tierkubectl apply-f child-dev-large.yaml# 3. Move workloads at your own pace, then delete the old cluster.#    Existing clusters keep the resolution snapshot they were built from;#    KubeCell never mutates a running child cluster in place.

Why no in-place resize

Quota changes ripple into the Foundation Work, the child control-plane size, and the resolution snapshot. Recreating keeps every step observable and reversible; patching would make partial failure states hard to reason about.

Credentials and endpoint changes

The child admin kubeconfig is always published. When the discovered host address changes, the controller rewrites and republishes the Secret — clients should re-fetch rather than pin an address.

credentials.sh

# Find the published credentialkubectl-n kubecell-system get vc child-dev \  -o jsonpath='{.status.credential.secretName}{"\n"}'# Endpoint changed? The controller republishes the kubeconfig.# Re-fetch the Secret rather than editing the file by hand.kubectl-n kubecell-system get secret <secret-name> \  -o jsonpath='{.data.kubeconfig\.yaml}' | base64 -d > child-dev.yamlchmod600 child-dev.yaml

Credentials are cluster-admin

Child kubeconfigs grant admin on the child cluster. Distribute them through your secret manager, keep file modes at 600, and revoke by deleting the VirtualCluster.

Ingress and DNS

Host Traefik owns ports 80 and 443 for the entire host. It watches only VirtualCluster namespaces carrying the KubeCell label. Developers choose the Ingress name; the rest of the hostname is derived.

ingress.yaml

# In the child cluster: standard Ingress, class kubecellapiVersion: networking.k8s.io/v1kind: Ingressmetadata:  name: web  namespace: defaultspec:  ingressClassName: kubecell  rules:  - host: web.child-dev.apps.example.com   # <ingress>.<vc>.<apps-suffix>    http:      paths:      - path: /        pathType: Prefix        backend:          service:            name: web            port:              number: 80

DNS automation is enabled when a second host arrives: point a wildcard record for *.<vc>.<apps-suffix> at the Cell's ingress address. Ingress routing is outside the locked-combination commitments, so verify it in your environment before promising it to tenants.

Storage lifecycle

Child PVCs are reflected to host PVCs on TopoLVM. If a child PVC does not bind, check topology labels first, then TopoLVM health.
Under Retain, deleting a VirtualCluster leaves PVs Released and data intact. Reclaim deliberately with lvremove for cluster-created volumes only; never remove the volume group.
Storage free space is read from TopoLVM/lvmd. Inferring it from PV capacity is explicitly forbidden, so treat freeCapacityKnown=false as "unknown", not "zero".

Upgrades and version pinning

Versions move through releases. The controller reports observed versions; it never upgrades anything on its own.

upgrade.sh

# Version drift is reported, not silently fixedkubectl-n kubecell-system get cell cell1 \  -o jsonpath='{.status.provider.k3kVersion}{"\n"}'# Upgrades move through releases:#   1. Build/verify a new release bundle (digests + pinned versions).#   2. doctor against the current fleet.#   3. Apply the new management chart, then the host baseline per Cell.#   4. Watch Cell conditions and recreate child clusters only if the#      release notes require it.

Planned host maintenance

The first version does not migrate running child clusters between hosts. Plan maintenance windows accordingly and communicate them to tenants.

maintenance.sh

# Planned host maintenance (v1 has no live migration)# 1. Stop creating new child clusters on the Cell (tell tenants, or#    temporarily remove the class from your self-service flow).# 2. Delete or migrate child clusters deliberately; do not drain the host#    node under running reflected Pods and expect a graceful move.# 3. Perform maintenance, re-apply labels, verify the lease renews.# 4. Confirm inventory freshness before reopening the Cell.

Cleanup checklist

Delete child clusters you no longer need; the finalizer cleans the host side.
Check for Released PVs and decide reclaim vs. retain per volume.
Remove unused VirtualNodeClasses so tenants cannot request dead tiers.
Remove the OCM ManagedCluster registration explicitly if a host leaves the fleet; Helm uninstall does not do it for you.

Next: API reference or troubleshooting.

Host onboarding

API reference