> ## Documentation Index
> Fetch the complete documentation index at: https://taskdaemon.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Prometheus

# Prometheus Metrics

TaskDaemon exposes Prometheus metrics for monitoring.

## Metrics Endpoint

```
http://localhost:8080/metrics
```

## Available Metrics

| Metric                             | Type      | Description                        |
| ---------------------------------- | --------- | ---------------------------------- |
| `taskdaemon_tasks_queued_total`    | Counter   | Total tasks queued                 |
| `taskdaemon_tasks_completed_total` | Counter   | Total tasks completed successfully |
| `taskdaemon_tasks_failed_total`    | Counter   | Total tasks failed                 |
| `taskdaemon_task_duration_seconds` | Histogram | Task processing duration           |
| `taskdaemon_queue_size`            | Gauge     | Current queue size                 |
| `taskdaemon_active_workers`        | Gauge     | Number of active workers           |

## Prometheus Configuration

Create `prometheus.yml`:

```yaml theme={null}
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'taskdaemon'
    static_configs:
      - targets: ['taskdaemon:8080']
```

## Docker Compose Setup

```yaml theme={null}
services:
  taskdaemon:
    image: mshelia/taskdaemon
    ports:
      - "8080:8080"
    volumes:
      - ./handlers.toml:/app/handlers.toml
      - /var/run/docker.sock:/var/run/docker.sock

  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    depends_on:
      - taskdaemon
```

## Useful Queries

### Throughput

```promql theme={null}
# Tasks completed per second
rate(taskdaemon_tasks_completed_total[1m])

# Tasks queued per second
rate(taskdaemon_tasks_queued_total[1m])
```

### Latency

```promql theme={null}
# Average task duration
rate(taskdaemon_task_duration_seconds_sum[5m]) / rate(taskdaemon_task_duration_seconds_count[5m])

# 95th percentile duration
histogram_quantile(0.95, rate(taskdaemon_task_duration_seconds_bucket[5m]))
```

### Error Rate

```promql theme={null}
# Failure rate
rate(taskdaemon_tasks_failed_total[5m]) / rate(taskdaemon_tasks_queued_total[5m])

# Success rate
rate(taskdaemon_tasks_completed_total[5m]) / rate(taskdaemon_tasks_queued_total[5m])
```

### Queue Health

```promql theme={null}
# Current queue depth
taskdaemon_queue_size

# Queue growth rate
deriv(taskdaemon_queue_size[5m])
```

## Alerting Rules

Example Prometheus alerting rules:

```yaml theme={null}
groups:
  - name: taskdaemon
    rules:
      - alert: HighQueueDepth
        expr: taskdaemon_queue_size > 1000
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "TaskDaemon queue depth is high"

      - alert: HighErrorRate
        expr: rate(taskdaemon_tasks_failed_total[5m]) / rate(taskdaemon_tasks_queued_total[5m]) > 0.1
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "TaskDaemon error rate above 10%"

      - alert: SlowTasks
        expr: histogram_quantile(0.95, rate(taskdaemon_task_duration_seconds_bucket[5m])) > 30
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "95th percentile task duration above 30s"
```