PyQGIS Engineering & Automation

Running a Nightly PyQGIS Job with cron

Schedule a PyQGIS job with cron so it actually runs: a wrapper that sets the environment explicitly, an flock guard against overlapping runs, a timeout…

TL;DR: never put the Python command directly in the crontab — put a wrapper script there that sets the environment explicitly, takes a lock, runs the job and exits with a meaningful code, and resolve every path from the script’s own location rather than the working directory. This page is part of the scheduled batch processing and pipeline orchestration guide.

Complete Runnable Setup

The crontab entry is one line and does nothing except invoke the wrapper:

text
# m h  dom mon dow   command
  15 3  *   *   *    /opt/pipelines/nightly/run.sh >> /var/log/nightly.log 2>&1

The wrapper is where all the environment work happens:

bash
#!/usr/bin/env bash
# /opt/pipelines/nightly/run.sh — everything cron does not give us
set -euo pipefail

HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

# 1. the environment a login shell would have provided
export PATH="/usr/bin:/bin:/usr/local/bin"
export PROJ_LIB="/usr/share/proj"
export GDAL_DATA="/usr/share/gdal"
export QT_QPA_PLATFORM="offscreen"
export PYTHONPATH="/usr/share/qgis/python:${PYTHONPATH:-}"

# 2. one run at a time; a second invocation exits quietly rather than colliding
exec 9>"/var/lock/nightly.lock"
if ! flock -n 9; then
    echo "$(date -Is) another run is still in progress; skipping"
    exit 0
fi

# 3. run the job from a known directory, with a hard time limit
cd "$HERE"
timeout 3h /usr/bin/python3 "$HERE/nightly.py" --config "$HERE/config.toml"
status=$?

# 4. a code the scheduler and any monitoring can act on
if [ $status -eq 124 ]; then
    echo "$(date -Is) job exceeded its time limit"
fi
exit $status
The wrapper that makes a cron entry reliable A four-stage wrapper: set the environment explicitly, take an exclusive lock, run the job with output captured, and exit with a code the scheduler can act on. set env explicitly take the lock or exit quietly run the job output captured exit code acted on then if free always

Architecture Breakdown

The environment is the whole problem

A cron job runs with a nearly empty environment: a minimal PATH, no variables from the user’s profile, and the home directory as its working directory. Everything a login shell would have arranged has to be arranged explicitly.

What cron does not give your process A grid of four things an interactive shell provides that a cron job does not — a full PATH, the QGIS environment variables, a working directory and a virtual environment — with the symptom each absence produces. under cron symptom PATH a minimal default command not found PROJ_LIB / GDAL_DATA unset coordinates silently wrong working directory the home directory relative paths miss virtualenv not activated ImportError on qgis

The second row of that table is the dangerous one. A missing PROJ_LIB does not raise — PROJ falls back to whatever it can find, and coordinate transformations quietly lose accuracy or pick a different datum operation. A job that runs nightly for a month with the wrong grid produces a month of subtly wrong output that nobody noticed because nothing failed.

flock for one run at a time

exec 9>lockfile opens a file descriptor for the life of the script and flock -n 9 takes an exclusive lock on it without waiting. If another run holds it, the second exits immediately with status 0 — a skipped run is not a failure and should not page anyone.

Because the lock is held by the file descriptor, the kernel releases it when the process dies for any reason. A job killed by the out-of-memory killer leaves no stale lock behind, which is the main failure mode of lock files implemented by writing a PID.

timeout as a backstop

Any job that can hang should have an upper bound, and timeout provides one for free. The exit code 124 is specifically “the time limit was reached”, which is worth distinguishing in the log from an ordinary failure — a job that is failing and a job that is getting slower need different responses.

Diagnosing “It Works When I Run It”

The gap between an interactive run and a scheduled one is narrow enough to enumerate, which makes this class of failure quick to resolve once you know where to look.

The job runs by hand but not from cron A decision tree over the three usual causes: an environment variable that only the login shell sets, a relative path resolved against the wrong directory, and a permission difference between the interactive user and the cron user. It works in my shell and fails at 03:00 ImportError environment set it in the wrapper file not found relative path resolve from __file__ permission denied different user check ownership

Nearly every cron-specific failure is one of three things, and each has a quick test.

For an import or command failure, run the job with a deliberately stripped environment — env -i /bin/bash --noprofile --norc — and see whether it reproduces. That is much closer to what cron does than your interactive shell.

For a missing file, look for a relative path. Under cron the working directory is the user’s home, so open("config.toml") reads a file that exists somewhere else entirely. Resolving paths from the script’s own location, as the wrapper does with $HERE, removes the whole class of problem.

For permission errors, check which user the crontab belongs to. A job in /etc/cron.d may run as root or as a service account with no access to the developer’s directories, and the resulting error names a file rather than the actual cause.

Making the Log Worth Reading

Redirecting to a file is the minimum. What makes the log useful afterwards is one structured line per run, written whatever the outcome:

python
import logging
import time

log = logging.getLogger("nightly")


def run_and_report(work) -> int:
    """Run `work`, always emitting one summary line the log can be grepped for."""
    started = time.monotonic()
    produced = failed = 0
    try:
        produced, failed = work()
        return 0 if failed == 0 else 1
    finally:
        log.info("summary run=nightly produced=%d failed=%d seconds=%.1f",
                 produced, failed, time.monotonic() - started)

A month of those lines answers “when did this start getting slower” and “how often does it actually fail” without any additional tooling, which is far more than most scheduled jobs can say.

Production Best Practices

  • Put a wrapper in the crontab, never a bare Python command.
  • Set every environment variable explicitly, including PROJ_LIB and GDAL_DATA.
  • Resolve paths from the script location, never from the working directory.
  • Take a lock, and exit 0 when another run holds it.
  • Bound the runtime with timeout, and treat 124 as its own outcome.
  • Rotate the log. An append-only redirect will eventually fill the disk, and it will do so on a weekend.

Frequently Asked Questions

Should I use a systemd timer instead?

If the machine has systemd, usually yes. Timers give you journal logging with no redirect, OnFailure= hooks for alerting, Restart= policies, and dependency ordering against other units. The service unit also carries the environment declaratively, so the wrapper shrinks to the job itself.

Cron remains a perfectly good answer for a single job on a machine you do not control, and the patterns above — explicit environment, lock, timeout, exit codes — apply to both.

How do I get failure notifications?

Cron mails the output of a failing job to the crontab owner if a mail transport agent is configured, which on modern servers it usually is not. The reliable approach is for the wrapper to send the notification itself on a non-zero exit — a webhook to a chat channel is a couple of lines of curl — or to run under systemd, where OnFailure= can trigger a notification unit.

Whichever you choose, test it by making the job fail deliberately. An untested alert path is indistinguishable from no alert path.

Can I run the job inside a container from cron?

Yes, and it is often the better shape: the crontab invokes docker run and the container carries the whole environment, which removes most of the wrapper’s job. Keep the lock outside the container, on the host, since two containers know nothing about each other.

Mount the data and output directories rather than baking them in, and be explicit about the user inside the container so output files do not end up owned by root.

What about the QGIS user profile?

A headless job still resolves a profile directory, and under cron the home directory may differ from the one you tested with. Set QGIS_CUSTOM_CONFIG_PATH to a directory the job owns, so the profile — including any authentication database — is predictable rather than inherited.

How do I stop overlapping runs across two machines?

flock is per host, so a job scheduled on two machines needs a shared lock: a row in a database, a lock object in shared storage, or simply not scheduling the same job twice. The last is usually right — high availability for a nightly batch is rarely worth the coordination problem it creates.

Should the job email its output?

Prefer writing a structured log and letting something else decide what to send. Mail from cron is all-or-nothing and quickly becomes noise, and noise is how the one message that mattered gets missed. One summary line per run in a log that is monitored is worth more than a nightly mail nobody opens.