Docs / Kubernetes

Troubleshooting the Kubernetes agent

No data, offline agents, RBAC gaps, network egress, kubelet stats and resource limits.

kubernetestroubleshootingrbacegressofflinehelm

No cluster appears on the Kubernetes page

  1. Check the agent pods: kubectl get pods -n verops. You should see the cluster-agent pod plus one node-agent pod per node, all Running.
  2. If pods are missing, re-check the Helm install (helm status verops-agent -n verops). If they're crash-looping, read the logs: kubectl logs -n verops <pod>.
  3. Authentication — repeated 401s in the log mean the API key is wrong or revoked. Keys start with ta_live_ and are shown once at creation (Settings → API Keys). Fix with: helm upgrade verops-agent verops/verops-k8s-agent -n verops --reuse-values --set apiKey=<new key>
  4. Network egress — the agent only needs outbound HTTPS (443) to your backendUrl. Connection-refused/timeout logs point to egress NetworkPolicies, firewalls or a proxy blocking app.verops.io. Allow that host (or set endpoint to your permitted ingest URL).

Data appears about a minute after the first successful ship; the page refreshes every 20 seconds.

Cluster shows as offline or goes stale

The agent heartbeats every 30 seconds; the platform flips it offline after roughly three missed beats (~90 seconds). If a previously healthy cluster goes offline: check the cluster-agent pod for restarts or eviction, then egress connectivity. Data resumes automatically once the agent reconnects.

Tabs are partially empty (RBAC)

The chart installs a read-only ClusterRole (get/list/watch on nodes, pods, namespaces, services, endpoints, deployments and other workloads, jobs, HPAs, ingresses, network policies, persistent volumes/claims, resource quotas, config maps and events). If you set rbac.create=false, your own binding must cover the same list — Forbidden errors in the agent log and missing tabs (e.g. Storage empty, Events empty) are the symptom. Fix by re-enabling rbac.create=true or extending your ClusterRole.

Nodes and pods listed, but CPU/memory show 0

Inventory comes from the cluster agent; usage comes from the node DaemonSet reading each kubelet. Check that the node-agent pods are running on every node (nodeAgent.enabled=true, and the DaemonSet tolerates your taints — the default tolerations land on all nodes) and that RBAC includes nodes/stats and nodes/proxy (the chart's default role does).

Agent pods OOMKilled or throttled on large clusters

Defaults are modest (50m/64Mi requests, 500m/256Mi limits). On clusters with thousands of objects, raise them for the cluster agent:

helm upgrade verops-agent verops/verops-k8s-agent -n verops --reuse-values \
  --set clusterAgent.resources.limits.memory=512Mi

OTel-sourced cluster: tabs empty or missing data

  • Events tab empty — the k8sobjects receiver isn't configured in your collector.
  • Services/deployments missing — the collector isn't scraping kube-state-metrics, or kube-state-metrics isn't installed in the cluster at all. Install it from the prometheus-community Helm chart (for example into a metrics namespace) and point the collector's prometheus receiver at it; the k8s_cluster and k8sobjects receivers keep working without it, which is why the cluster still appears.
  • No node/pod usage — the kubeletstats receiver isn't running, or your collector version emits older metric names than the current OTel semantic conventions.
Still stuck? The Data Explorer is a quick truth test: if SELECT count() FROM metrics WHERE metric_name LIKE 'k8s_%' over the last 15 minutes returns 0, nothing is arriving (installation problem); if it returns rows but the page is empty, contact support with your cluster name.