Skip to content
Cluster-Wide OPcache Invalidation

Cluster-Wide OPcache Invalidation

Since ePHPm 0.4.0, one ephpm deploy command atomically invalidates a vhost’s OPcache on every node of a cluster — no restarts, no rolling deploys, no opcache.validate_timestamps stat cost on the hot path. The deploy event is a single write to the gossip-replicated KV store; each node’s per-request watcher (one atomic load + one KV read) drops the vhost’s cached scripts before the next request executes.

Deploys are events (ephpm serve)

Starting in v0.5.0, ephpm serve defaults opcache.validate_timestamps=0: the server trusts the OPcache and never stat()s cached scripts on the hot path. Code changes become a deliberate event — they go live only when you run ephpm deploy (or ephpm cache reset), which invalidates the cache through the RESP listener. ephpm dev keeps validate_timestamps=1 so the edit-refresh loop stays instant.

Modevalidate_timestamps defaultCode changes go live…
ephpm serve0 (off)on ephpm deploy / ephpm cache reset
ephpm dev (or bare ephpm)1 (on)on the next request after you save

Override either way with the [php] opcache_validate_timestamps knob (true re-enables stat-on-use under serve, e.g. for a bind-mounted docroot; false freezes the cache under dev). [php] opcache_revalidate_freq tunes how often a re-stat happens when validation is on.

⚠️ Serve mode + validation off needs an invalidation lever. With validate_timestamps=0, the only way to refresh cached code without a restart is ephpm deploy / ephpm cache reset, and those write over the RESP listener. If [kv.redis_compat] enabled = false, there is no lever at all — startup logs a WARN and cached code stays frozen until the process restarts. Either enable the RESP listener (loopback is fine) or set [php] opcache_validate_timestamps = true.

Why off and not just a high revalidate_freq? On a 500-file autoload app (ePHPm-lab bench #5), validate_timestamps=1 revalidate_freq=2 measured 912 rps, revalidate_freq=60 measured 999 rps (+9.5%), and off measured 995 rps. Since freq=60 already recovers essentially all the throughput, off is chosen for the deterministic deploys-are-events contract, not for raw speed — with a bonus that on container/overlay/network filesystems the stat() savings are larger, and in immutable-container deploys (files can’t change) validation is pure waste.

Resource-aware autotuning (ephpm serve)

The deploys-are-events model has a sibling: right-size the runtime to the box it lands on. A deploy is an event, and so is the pod it lands in — a 320 MiB / 0.25-CPU sidecar and a 4 GiB / 4-CPU node run the same image but should not run the same php.ini. Hand-tuning OPcache SHM, the per-request memory_limit, and buffer sizes per environment is exactly the kind of drift that immutable images are supposed to kill.

On boot, ephpm serve reads the container’s CPU quota and memory limit the same cgroup-aware way worker_count already does (cgroup v2 memory.max → v1 memory.limit_in_bytes/proc/meminfo MemTotal), then derives a tuned profile:

  • OPcache SHM (opcache.memory_consumption) tracks ~18% of the memory budget, clamped [64, 512] MB — big enough for large frameworks, never starving a tiny pod’s page cache. (On Windows the ceiling is 256 MB instead, because the segment is pagefile-backed and commit-charged in full at startup — see the config reference.)
  • Per-request memory_limit is (budget − opcache_shm − ~64 MB overhead) / worker_count, floored at 128 MB, so N concurrent requests can’t collectively exceed the pod’s cgroup limit and get OOM-killed.
  • Interned-strings and JIT buffers scale with the SHM / budget (clamped). Whether the JIT uses the buffer is shaped by tenancy ([php] opcache_jit): disable by default in every mode, including the multi-vhost deployments this guide covers — here because per-vhost opcache_invalidate never reclaims JIT buffer, so deploy churn would silently exhaust it, and in single-site serve because of the unfixed upstream tracing-JIT crash (#365). Forcing it on is an explicit, warned-about opt-in; watch ephpm_opcache_jit_buffer_free_bytes if you do. See OPcache JIT.
  • realpath_cache_size=16M / ttl=600 and zend.assertions=-1 (compiled out) are the standard production values; dev keeps PHP-friendly defaults.

Every directive resolves through explicit [php] config → derived → PHP default, so you can pin exactly the one knob you care about (opcache_memory_consumption = 256) and let the rest auto-tune, with ini_overrides as the final escape hatch. Serve startup logs one INFO line showing what was detected and derived (pinned values marked *):

autotune (serve): cpu_quota=0.25 mem=320MiB (cgroup v2) -> workers=1[cgroup_quota] opcache.memory_consumption=64MB memory_limit=192M interned=8MB jit_buffer=32MB (jit=disable) max_files=20000 realpath=16M/ttl=600 validate_timestamps=0 assertions=-1

See the config reference for the full formula table and every knob.

Requirements

  • ePHPm ≥ 0.4.0, [php] mode = "fpm" (the default — see gaps for worker mode)
  • [cluster] enabled = true for multi-node fan-out (single node works too — ephpm deploy then only affects that node)
  • [kv.redis_compat] enabled = true — the CLI writes the version key over the RESP listener (bind it to loopback)

Configuration

[cluster]
enabled = true
bind = "10.0.1.5:7946"          # this node's reachable address
join = ["10.0.1.6:7946"]        # any peer(s)
secret = "…"                     # seals gossip + KV data plane

[kv.redis_compat]
enabled = true
listen = "127.0.0.1:6379"       # loopback: only the local CLI needs it

[opcache]
cluster_invalidation = true      # default: true when [cluster] enabled

Deploying

# Invalidate one vhost, cluster-wide
ephpm deploy --site blog

# Single-site mode (no sites_dir): the default vhost
ephpm deploy

# Every vhost at once (broadcast key)
ephpm deploy --all

# Record a revision string alongside (observability only)
ephpm deploy --site blog --rev "$(git rev-parse --short HEAD)"

# Same wire effect, distinct name for dev/audit-log clarity
ephpm cache reset --site blog

Run it on any node; gossip converges to peers in ~1–3 seconds. Each node invalidates lazily on its next request for that vhost — a node serving no traffic pays nothing.

Verifying

opcache_invalidate() keeps entries listed in opcache_get_status()['scripts'], so presence-in-list is not a valid check. Use opcache_is_script_cached():

<?php // status.php
echo json_encode([
    'cached' => opcache_is_script_cached($_SERVER['DOCUMENT_ROOT'] . '/index.php'),
]);

Warm a script, run ephpm deploy, and the next request on every node reports cached: false, then true again once re-warmed. The ephpm_opcache_invalidations_total{vhost,trigger} counter increments on each node that performed an invalidation.

Kubernetes

A two-node StatefulSet needs three things beyond the config above: bind the pod IP (gossip advertises the bind address), use the headless-service pod DNS names as seeds, and run the CLI via kubectl exec:

env:
  - name: POD_IP
    valueFrom: { fieldRef: { fieldPath: status.podIP } }
# start script: bind = "${POD_IP}:7946"
# join = ["app-0.app-hs.ns.svc.cluster.local:7946", "app-1.app-hs.ns.svc.cluster.local:7946"]
kubectl exec app-0 -- ephpm deploy --site blog

A complete, tested manifest pair (demo + php-fpm comparison benchmark) lives in the ePHPm-lab repository under k8s/opcache-cluster.yaml.

Why not rely on opcache.validate_timestamps?

Timestamp validation pays a stat() per include site per request (bounded by revalidate_freq), on every node — and still can’t make a deploy atomic across a cluster. That is why serve mode turns it off by default (v0.5.0, see above) and uses the KV-driven invalidation event instead. The comparison table and alternatives are in the design document, along with the planned Phase 2 (per-vhost preload) and Phase 3 (file watcher).

Gaps and caveats

  • Worker mode is not wired yet — with [php] mode = "worker" the watcher is skipped and startup logs a WARN. Planned.
  • INCR does not replicate — SET, DEL, and EXPIRE all fan out (deletes ride a write-stamped gossip tombstone; EXPIRE re-emits the value with the new expiry). Read-modify-write ops like INCR are still local-only, so rate-limit middleware counters are per-node in a cluster (startup warns when this combination is active). See the clustered-kv-v2 roadmap for the owner-routed INCR design.
  • ephpm cache status is not implemented; use opcache_get_status().