Post

Monitoring Proxmox with Prometheus and Grafana

This guide walks you through setting up comprehensive monitoring for your Proxmox server and all its VMs using Prometheus and Grafana.

Posted
By Steve Bentley
7 min read
Monitoring Proxmox with Prometheus and Grafana

Architecture Overview

  • Proxmox VE Exporter: Collects metrics from Proxmox API (runs on monitoring VM)
  • Node Exporter: Collects system metrics from Proxmox host and VMs
  • Prometheus: Stores and queries metrics (runs on monitoring VM)
  • Grafana: Visualizes metrics with dashboards (runs on monitoring VM)

Prerequisites

  • Proxmox VE server (version 6.x or 7.x+)
  • A dedicated VM for monitoring stack (Ubuntu/Debian recommended, 2 CPU cores, 4GB RAM, 20GB disk)
  • Basic Linux command line knowledge
  • Root access to Proxmox host and monitoring VM

Setup Overview

On Proxmox Host: Only Node Exporter to collect host metrics

On Monitoring VM: Prometheus, Grafana, PVE Exporter, and Node Exporter

On Other VMs (optional): Node Exporter for per-VM metrics

Part 1: Install Node Exporter on Proxmox Host

On Proxmox host

1.1 Download and Install

1
2
3
4
5
6

cd /tmp
wget https://github.com/prometheus/node_exporter/releases/download/vX/node_exporter-X.linux-amd64.tar.gz
tar xvf node_exporter-X.linux-amd64.tar.gz
sudo cp node_exporter-X.linux-amd64/node_exporter /usr/local/bin/
sudo useradd --no-create-home --shell /bin/false node_exporter
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Make sure to get the latest version from Prometheus

1.2 Create Systemd Service

Create /etc/systemd/system/node_exporter.service:

1
2
3
4
5
6
7
8
9
10
11
12
13

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

1.3 Start Node Exporter

1
2
3
4

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
sudo systemctl status node_exporter

1.4 Verify It’s Running

1

curl http://localhost:9100/metrics

You should see metrics output. The Proxmox host setup is now complete!

Part 2: Create Monitoring User in Proxmox

On the Proxmox Web GUI:

  1. Go to Datacenter → Permissions → Users
  2. Click “Add” and create user:Prometheus
  3. Set Realm to Proxmox VE authentication server
  4. Set a strong password (you’ll need this later)
  5. Go to Permissions → Add → User Permission
  6. Path: /
  7. User: prometheus@pve
  8. Role: PVEAuditor
  9. Click “Add”

Part 3: Setup Monitoring VM

From this point forward, SSH into your monitoring VM and run all commands there.

3.1 Install Prometheus

Create Prometheus User

1

sudo useradd --no-create-home --shell /bin/false prometheus

Download and Install Prometheus

1
2
3
4
5
6
7
8
9

cd /tmp
wget https://github.com/prometheus/prometheus/releases/download/vX/prometheus-X.linux-amd64.tar.gz
tar xvf prometheus-X.linux-amd64.tar.gz
cd prometheus-X.linux-amd64

sudo cp prometheus /usr/local/bin/
sudo cp promtool /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool

Replace X with the latest version from Prometheus

Create Directories

1
2
3
4

sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
sudo chown prometheus:prometheus /etc/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus

Create Configuration Files

1
2
3
4

sudo touch /etc/prometheus/consoles
sudo touch /etc/prometheus/console_libraries
sudo chown -R prometheus:prometheus /etc/prometheus/consoles
sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries

3.2 Install Node Exporter on Monitoring VM

Download and Install

1
2
3
4
5
6

cd /tmp
wget https://github.com/prometheus/node_exporter/releases/download/vX/node_exporter-X.linux-amd64.tar.gz
tar xvf node_exporter-X.linux-amd64.tar.gz
sudo cp node_exporter-X.linux-amd64/node_exporter /usr/local/bin/
sudo useradd --no-create-home --shell /bin/false node_exporter
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Make sure to get the latest version from Prometheus

Create Systemd Service

Create /etc/systemd/system/node_exporter.service:

1
2
3
4
5
6
7
8
9
10
11
12
13

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

Start Node Exporter

1
2
3
4

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
sudo systemctl status node_exporter

3.3 Install Proxmox VE Exporter

Install Dependencies

1
2

sudo apt update
sudo apt install -y python3-pip python3-venv

Install PVE Exporter

1

sudo pip3 install prometheus-pve-exporter

Create Configuration File

Create /etc/prometheus/pve.yml

1
2
3
4
5
6

default:
  user: prometheus@pve
  password: your_password_here
  # Replace with your Proxmox host IP or hostname
  target: https://192.168.1.100:8006
  verify_ssl: false

Important: Replace your_password_here with the password you set for the prometheus@pve user, and replace 192.168.1.100 with your actual Proxmox host IP address.

Set Permissions

1
2

sudo chown prometheus:prometheus /etc/prometheus/pve.yml
sudo chmod 600 /etc/prometheus/pve.yml

Create Systemd Service for PVE Exporter

Create /etc/systemd/system/pve_exporter.service:

1
2
3
4
5
6
7
8
9
10
11
12

[Unit]
Description=Proxmox VE Exporter
After=network.target

[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/bin/pve_exporter /etc/prometheus/pve.yml
Restart=on-failure

[Install]
WantedBy=multi-user.target

Start PVE Exporter

1
2
3
4

sudo systemctl daemon-reload
sudo systemctl start pve_exporter
sudo systemctl enable pve_exporter
sudo systemctl status pve_exporter

Verify It’s Working

1

curl http://localhost:9221/pve

You should see Proxmox metrics. If you get connection errors, check:

  • The Proxmox host IP in /etc/prometheus/pve.yml is correct
  • The monitoring VM can reach the Proxmox host on port 8006
  • The prometheus@pve user credentials are correct

3.4 Configure Prometheus

Create /etc/prometheus/prometheus.yml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  # Prometheus itself
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  # Monitoring VM metrics
  - job_name: 'monitoring-vm'
    static_configs:
      - targets: ['localhost:9100']
        labels:
          instance: 'monitoring-vm'

  # Proxmox host metrics
  - job_name: 'proxmox-host'
    static_configs:
      - targets: ['192.168.1.100:9100']  # Replace with your Proxmox host IP
        labels:
          instance: 'proxmox-host'

  # Proxmox VE metrics (VMs, containers, storage, etc.)
  - job_name: 'proxmox'
    static_configs:
      - targets:
        - 'localhost:9221'
    metrics_path: /pve
    params:
      target: ['192.168.1.100'] # Replace with your Proxmox host IP
      module: [default]

  # Other VMs (optional - add after installing node_exporter on them)
  - job_name: 'other-vms'
    static_configs:
      - targets: 
        - 'vm1-ip:9100'
        - 'vm2-ip:9100'
        labels:
          group: 'vms'

Important: Replace 192.168.1.100 with your actual Proxmox host IP address.

Set Permissions

1

sudo chown prometheus:prometheus /etc/prometheus/prometheus.yml

Create Prometheus Systemd Service

Create /etc/systemd/system/prometheus.service:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus/ \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

Start Prometheus

1
2
3
4

sudo systemctl daemon-reload
sudo systemctl start prometheus
sudo systemctl enable prometheus
sudo systemctl status prometheus

Verify Prometheus is running: http://your-monitoring-vm-ip:9090

3.5 Install Grafana

Install Grafana

1
2
3
4
5

sudo apt-get install -y software-properties-common
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt-get update
sudo apt-get install grafana

Start Grafana

1
2
3

sudo systemctl start grafana-server
sudo systemctl enable grafana-server
sudo systemctl status grafana-server

Access Grafana

Open http://your-monitoring-vm-ip:3000

  • Default username: admin
  • Default password: admin
  • Change password on first login

Add Prometheus Data Source

  1. Click Settings (gear icon) → Data Sources
  2. Click “Add data source”
  3. Select “Prometheus”
  4. Set URL: http://localhost:9090
  5. Click “Save & Test”

Part 4: Install Node Exporter on Other VMs (Optional)

For each additional VM you want detailed metrics from, SSH into it and run:

1
2
3
4
5
6

cd /tmp
wget https://github.com/prometheus/node_exporter/releases/download/vX/node_exporter-X.linux-amd64.tar.gz
tar xvf node_exporter-X.linux-amd64.tar.gz
sudo cp node_exporter-X.linux-amd64/node_exporter /usr/local/bin/
sudo useradd --no-create-home --shell /bin/false node_exporter
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Make sure to get the latest version from Prometheus

Create Systemd Service

Create /etc/systemd/system/node_exporter.service:

1
2
3
4
5
6
7
8
9
10
11
12
13

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

Start Node Exporter

1
2
3
4

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
sudo systemctl status node_exporter

Verify It’s Running

1

curl http://localhost:9100/metrics

Part 5: Import Grafana Dashboards

RUN IN GRAFANA WEB INTERFACE (on monitoring VM)

  1. Click “+” → Import
  2. Enter dashboard ID: 10347 (Proxmox via Prometheus)
  3. Select your Prometheus data source
  4. Click “Import”

Import Node Exporter Dashboard

  1. Click “+” → Import
  2. Enter dashboard ID: 1860 (Node Exporter Full)
  3. Select your Prometheus data source
  4. Click “Import”

Part 5: Verification

5.1 Check Exporters

1
2
3

# Check if exporters are responding
curl http://localhost:9100/metrics  # Node Exporter
curl http://localhost:9221/metrics  # PVE Exporter

5.2 Check Prometheus Targets

Go to http://your-proxmox-ip:9090/targets and verify all targets show as “UP”

5.3 Test Queries in Prometheus

Try these queries in Prometheus:

  • up - Shows which targets are up
  • node_cpu_seconds_total - CPU metrics
  • pve_cpu_usage_ratio - Proxmox CPU usage

Maintenance

Update Prometheus

1
2
3
4
5
6
7
8

sudo systemctl stop prometheus
# Download new version from GitHub
cd /tmp
wget https://github.com/prometheus/prometheus/releases/download/vX.X.X/prometheus-X.X.X.linux-amd64.tar.gz
tar xvf prometheus-X.X.X.linux-amd64.tar.gz
sudo cp prometheus-X.X.X.linux-amd64/prometheus /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo systemctl start prometheus

Backup Grafana Dashboards

1
2

# Grafana dashboards are stored in database
sudo cp /var/lib/grafana/grafana.db /backup/location/grafana-$(date +%Y%m%d).db

Quick Command Reference

Monitoring VM Commands

1
2
3
4
5
6
7
8
9
10
11
12

# Check all services
sudo systemctl status prometheus grafana-server pve_exporter node_exporter

# Restart services
sudo systemctl restart prometheus
sudo systemctl restart grafana-server
sudo systemctl restart pve_exporter

# View logs
sudo journalctl -u prometheus -f
sudo journalctl -u grafana-server -f
sudo journalctl -u pve_exporter -f

Proxmox Host Commands

1
2
3
4
5

# Check node_exporter
sudo systemctl status node_exporter

# View metrics
curl http://localhost:9100/metrics

Useful URLs

After setup, bookmark these:

  • Grafana: http://monitoring-vm-ip:3000
  • Prometheus: http://monitoring-vm-ip:9090
  • Prometheus Targets: http://monitoring-vm-ip:9090/targets
  • Proxmox Host Metrics: http://proxmox-ip:9100/metrics
  • PVE Exporter: http://monitoring-vm-ip:9221/pve
cybersecurity, monitoring, grafana, prometheus, proxmox, homelab
homelab monitoring
This post is licensed under CC BY 4.0 by the author.