Scheduling per-instance maintenance

From Canasta Wiki

canasta schedule runs a command inside one of an instance's containers on a recurring schedule. Typical uses are purging old parser cache entries, running a MediaWiki maintenance script every night, or housekeeping for a sidecar service. It works the same way on Docker Compose and Kubernetes.

Backups have their own scheduler, canasta backup schedule; see Scheduling recurring backups.

Adding a schedule

Give the schedule a name, a cron expression, and the command to run:

canasta schedule add -i mywiki -n parser-cache --cron '0 3 * * *' \
  --command 'php maintenance/run.php purgeParserCache --age 604800'
  • Name (-n, --name): lowercase letters, digits and hyphens. backup is reserved for the backup schedule.
  • Cron expression (--cron): exactly five fields, minute hour day-of-month month day-of-week. Shortcuts such as @daily are not accepted, and only letters, digits and * / , - are allowed in the fields.
  • Command (--command): runs under bash inside the container (or sh if the container has no bash), so paths, PHP and the environment are the container's. In the web container it runs from the MediaWiki directory as www-data, as maintenance scripts do. On a wiki farm, add --wiki=<id> to a maintenance script to target a wiki other than the first.
  • Service (-s, --service): the container to run in. The default is web. Any other service must be running in the instance, and add lists the running services if it isn't.

A name that is already in use is refused. To change a schedule, remove it and add it again.

Listing, running and removing

canasta schedule list -i mywiki

This shows each schedule's service, cron expression and command, plus the time and result of its last run: ok, exit N (the command's own exit status), or never. If a backup schedule is set, it appears as a read-only backup row. list also warns when the host's crontab doesn't match the instance's schedules, for example a schedule with no crontab entry.

To run a schedule now, whatever its cron expression says:

canasta schedule run -i mywiki -n parser-cache

This is the same command each scheduled firing runs. It prints the command's output and exits non-zero if the command fails, so it is also a quick way to test a new schedule.

To remove a schedule and its crontab entry:

canasta schedule remove -i mywiki -n parser-cache

A run that is already in progress finishes, and its log is kept.

How schedules run

Each schedule is a crontab entry on the host where the instance runs: the local host, the remote host of a remote Docker Compose instance, or the control-plane node for Kubernetes. This is the same host scheduled backups use. The entry runs canasta schedule run at the scheduled times, which reads the command from the instance's configuration and runs it in the container. The command itself never appears in the crontab.

That host needs the cron package and a canasta it can run. A local instance already has one. For a remote or Kubernetes instance, install canasta on the instance's host first:

canasta install canasta -H <host>

canasta schedule add stops with this instruction if canasta is missing on that host, or if the version installed there is too old to have canasta schedule.

A few things to know about runs:

  • Logs. Each run's output is appended to schedule-<name>.log in the instance directory. Nothing rotates these logs. Their state is kept in .schedule-state/, and both are left out of gitops.
  • Overlap. If a run is still going when the schedule fires again, the new run is skipped, and the log says so.
  • Restarts. A canasta restart or canasta upgrade that replaces the container stops a run in progress. That run is recorded as failed, and the next firing runs as usual.

Backups, restores and gitops

Schedules are stored in config/schedules.yml in the instance's configuration:

  • Backups capture the file, and finishing a restore re-creates the crontab entries from it, including on a new host or cluster.
  • On a gitops instance, add and remove stage the file, so schedules travel to the other hosts with the rest of the config. canasta gitops pull re-creates the crontab entries when the file changes.
  • Deleting an instance removes its schedule entries from the crontab.

Without further options, every gitops host runs each schedule against its own instance. To run a schedule only on some hosts, name them with --gitops-hosts:

canasta schedule add -i mywiki -n rebuild --cron '30 4 * * 0' \
  --command 'php maintenance/run.php rebuildall' --gitops-hosts prod

The names must be hosts in the repository's hosts/hosts.yaml. A schedule limited this way is installed only on those hosts, so a staging host, or a clone restored from a backup, does not run it. --gitops-hosts is refused on an instance that does not use gitops.

Every time the file is read, the whole file is checked again, so a malformed file that arrives through gitops is refused rather than half applied.

See also