Docker Deployment

TaskDaemon docker deployment guide

Docker Deployment

TaskDaemon launches handler containers through an existing Docker engine. Check docker info, build or pull configured handler images, then start the daemon.

Docker Run

docker run -d --name taskdaemon \
  -p 8080:8080 -p 50051:50051 \
  -e DAEMON_DB_PATH=/data/tasks.db \
  -e DAEMON_WORKERS=4 \
  -e DAEMON_BATCH_SIZE=5 \
  -v task-data:/data \
  -v "$(pwd)/handlers.toml:/app/handlers.toml:ro" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  mshelia/taskdaemon:0.2.0-alpha

Use a validated image tag or digest. Build the repository Dockerfile if you need the current source rather than an older published image.

Mount	Purpose
task-data:/data	Persist SQLite and its WAL files across container replacement
handlers.toml:/app/handlers.toml:ro	Handler configuration
/var/run/docker.sock:/var/run/docker.sock	Access to the existing Docker engine

Port 8080 serves HTTP, metrics, and health; 50051 serves plaintext gRPC. A process with access to the Docker socket can control that engine, so restrict access to the daemon container and its APIs.

Docker Compose

services:
  taskdaemon:
    image: mshelia/taskdaemon:0.2.0-alpha
    ports:
      - "8080:8080"
      - "50051:50051"
    volumes:
      - task-data:/data
      - ./handlers.toml:/app/handlers.toml:ro
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      DAEMON_DB_PATH: /data/tasks.db
      DAEMON_WORKERS: "4"
      DAEMON_BATCH_SIZE: "5"
      DAEMON_DB_MAX_CONNECTIONS: "20"
      DAEMON_LOG_LEVEL: info

volumes:
  task-data:

Four workers and 20 connections are a starting point for measurement, not a throughput guarantee. SQLite serializes writers; extra connections or workers can add contention. Match workers to useful handler concurrency and tune using the actual container workload.

docker compose down retains named volumes. docker compose down -v deletes them and the task history.

Build Handler Images First

docker build -t my-handler:latest ./handlers/my-handler
docker compose up -d taskdaemon

With Compose build definitions, run docker compose build for the handler services before starting TaskDaemon. depends_on alone does not establish handler readiness or guarantee that an image is built. TaskDaemon creates its own persistent handler containers from the configured images.

Startup Checks

The supplied daemon image includes curl. This health check inspects the JSON status because /health currently returns HTTP 200 even for an unhealthy response:

healthcheck:
  test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health | grep -q '"status":"healthy"'"]
  interval: 30s
  timeout: 10s
  retries: 3

/health checks the daemon and reads the exact Pending count from storage. Handler initialization has no readiness handshake. Submit a smoke task for every handler and wait for completed before sending production traffic. First-task latency can include runtime or model initialization.

Monitoring

Use the same taskdaemon service and named volume above. Add these services and the Prometheus configuration:

services:
  prometheus:
    image: prom/prometheus:latest
    profiles: ["monitoring"]
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro

  grafana:
    image: grafana/grafana:latest
    profiles: ["monitoring"]
    ports:
      - "3001:3000"
    environment:
      GF_SECURITY_ADMIN_PASSWORD: replace-with-your-password
    volumes:
      - grafana-data:/var/lib/grafana

volumes:
  grafana-data:

Merge this into your Compose file, then use docker compose --profile monitoring up -d. Configure Grafana to use http://prometheus:9090.

Recovery and Timeouts

Both SQLite and Hybrid persist claims, results, retry policy, and retry visibility. Hybrid caches task lookups while SQLite remains authoritative for queue operations. A healthy worker renews all its outstanding claims every 30 seconds, including tasks waiting in a batch. A Processing task whose lease is over five minutes old becomes eligible for recovery. Graceful shutdown releases unexecuted claims without spending retries.

Recovery provides at-least-once execution: use idempotent handlers or deduplicate external effects by task_id. See environment configuration for retry and timeout behavior.