BeginnerLesson 3 of 9

Data Sources (Prometheus)

Connect Grafana with Prometheus, validate queries, and troubleshoot common datasource issues.

Simple Explanation (ELI5)

Grafana needs a data source to draw charts. Prometheus is the most common one for metrics. Connect once, then all panels can query it.

Technical Explanation

A Grafana Prometheus datasource points to the Prometheus base URL. Grafana sends PromQL queries to that endpoint and renders results. In Kubernetes, this URL is often a service DNS name in the monitoring namespace.

Visual Section

Grafana
→
Prometheus API
→
Metrics Results

Hands-on Commands

yaml
apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus-server.monitoring.svc.cluster.local:9090
    isDefault: true
bash
# Prometheus health check
curl http://prometheus-server.monitoring.svc.cluster.local:9090/-/ready

# Grafana health check
curl http://localhost:3000/api/health

# Kubernetes port-forward test
kubectl -n monitoring port-forward svc/prometheus-server 9090:9090

Debugging Scenarios

Real-world Use Case

A team connects Grafana to HA Prometheus and uses one default datasource across all service dashboards for consistent metrics access.

Interview Questions

Beginner

What is a Grafana datasource?▾

A configured backend Grafana queries for visualization and alerts.

Why connect Grafana to Prometheus?▾

Prometheus provides time-series metrics that Grafana visualizes.

What protocol is typically used?▾

HTTP/HTTPS to the Prometheus API endpoint.

What does Test & Save do?▾

Validates datasource connectivity and basic query capability.

Can one dashboard use multiple data sources?▾

Yes, panels can point to different data sources.

Intermediate

How do you configure Grafana datasource as code?▾

Use provisioning files under Grafana provisioning/datasources.

Proxy mode vs direct mode?▾

Proxy mode routes queries via Grafana server; direct mode uses browser-to-datasource path.

Why datasource permissions matter?▾

To prevent unauthorized query access and accidental exposure.

How handle Prometheus URL in Kubernetes?▾

Use service DNS names and namespace-aware endpoints.

How validate datasource latency impact?▾

Measure query response times and panel render durations.

Scenario-based

Grafana says datasource down, but Prometheus UI works from your laptop. Why?▾

Grafana server network path differs from your laptop path.

Query returns data in Prometheus UI but no data in panel. What next?▾

Check panel datasource selection, time range, and variable filters.

TLS-enabled Prometheus fails in Grafana. Common fix?▾

Configure proper CA cert/auth settings and endpoint URL.

How would you switch datasource between staging and prod safely?▾

Use datasource variables and environment-specific provisioning.

Datasource works but dashboards are slow. Why?▾

Heavy PromQL queries, large time windows, and frequent refresh.

Summary

Prometheus integration is the foundation of practical Grafana usage. If datasource setup is solid, dashboard development becomes reliable and fast.