# Audit Log The audit log provides a tamper-evident record of content changes across the site. Every create, modify, delete, workflow transition, permission change, and working copy operation is captured, timestamped, and attributed to the acting user. Entries are stored in a dedicated Elasticsearch index with a cryptographic hash chain that allows verification of log integrity. ## Architecture The system follows an asynchronous pipeline to avoid adding latency to content operations: ``` Plone (event subscriber) | | after-commit hook v Redis Queue (low priority) | | RQ worker picks up job v Elasticsearch (dedicated audit log index) ``` 1. **Event subscribers** react to Zope lifecycle events (add, modify, delete), workflow transitions, local role changes, and working copy events. Subscribers collect a minimal entry dict synchronously. 2. **After-commit hook** — the entry is not enqueued immediately. Instead, a `transaction.addAfterCommitHook` callback ensures the entry is only sent to the queue when the ZODB transaction commits successfully. If the transaction is aborted, no audit entry is created. 3. **Redis Queue** — the entry is placed on the `low` priority RQ queue so that audit writes never compete with higher-priority indexing tasks. 4. **RQ worker** — the worker task acquires a Redis distributed lock (`audit_chain_lock`), reads the previous hash from Redis (`audit_chain_latest`), computes a SHA-256 hash chaining the new entry to the previous one, stores the entry in Elasticsearch, and updates the latest hash in Redis. In test mode (`ZOPETESTCASE=1`) the queue runs synchronously so entries are available immediately after commit. ## Tracked Actions The following actions are recorded: **`created`** A new Dexterity content object was added. The `details` dict records the values it was created with in a `changed_fields` mapping, in the same shape `modified` uses, skipping fields that hold no value. Suppressed during working copy operations (see [Working Copy Integration](#working-copy-integration)). **`modified`** One or more fields on a Dexterity object changed. The `details` dict contains a `changed_fields` mapping with the new value for each changed field. Complex field values are formatted compactly: files show filename and size, relations show the path to the target object. Rich text is stored in full (the raw source), so the diff view can show the complete change between two revisions. **`deleted`** A Dexterity object was removed or trashed. The `details` dict records the `original_path` of the deleted object. Trashing moves the object into the trash container, which renames it to a uuid, so for that path `original_path` comes from the trash annotation rather than from the object — its own physical path at that point is the old parent joined to the new id, which never existed. Suppressed during working copy operations. Content deleted via REST API or `api.content.delete` is soft-deleted (moved to trash) which fires `IObjectTrashedEvent`. Permanently removing content from the trash does **not** create a duplicate entry (objects with the `ITrashed` marker are skipped). **`moved`** A Dexterity object was moved to a different container. The `details` dict records the `old_path`; the new location is the entry's `content_path`. Only the moved object itself is recorded — the event is re-dispatched to every descendant, but sublocations are skipped so moving a folder produces a single entry. **`renamed`** The id of a Dexterity object changed while it stayed in the same container. Details include `old_id`, `new_id` and the `title`. Both rename forms let the title be changed together with the id, and both overwrite the title *before* the rename fires any event, so the previous title is no longer available — only the title the object ended up with is recorded. **`restored`** An object was restored from the trash. The entry's `content_path` is the location it was restored to; the original trash path is no longer available when the event fires, so `details` is empty. **`state_changed`** A workflow transition was executed. Details include the `transition` name, `new_state`, and `workflow` id. **`permission_changed`** Local roles on a content object were modified. Details include the full `local_roles` mapping (user/group to list of roles). **`property_changed`** The `layout` or `default_page` property on a content object was changed. These properties are set via OFS methods that do not fire `ObjectModifiedEvent`, so they are tracked via monkey patches (see [Property Change Tracking](#property-change-tracking)). Details include the `property` name plus `old` and `new` values. **`working_copy_created`** A working copy was created from a baseline object. The entry is recorded against the baseline. **`working_copy_applied`** A working copy was applied back to its baseline. The entry is recorded against the baseline. **`working_copy_discarded`** A working copy was discarded without applying. The entry is recorded against the baseline. `logged_in` / `logged_out` A user signed in or out. See [Login and Logout Tracking](#login-and-logout-tracking). Entries written while editing a working copy carry the **baseline's** UID in `parent_uid`, the same field that links a block to its page, so they show up on the baseline's object audit log. Without this they would only ever be reachable from the global log: applying a working copy copies field values without notifying anything, so the baseline gets no entries of its own, and the working copy is deleted afterwards, leaving its UID unresolvable. `content_uid` still identifies the edited object and `content_path` still records where the edit physically happened, so nothing is lost. Such entries carry `details["working_copy"]` and render with a badge — including entries from a working copy that was discarded and never applied. Folder reordering (`IContainerModifiedEvent`) is intentionally **not** tracked. The event fires for all container modifications and cannot reliably distinguish reordering from add/remove operations that are already tracked separately. Trashing and restoring are implemented as moves, but neither produces a `moved` or `renamed` entry: the trash direction is recognised by the trash container in the object's path, the restore direction by the `ITrashed` marker, which is only dropped once the restore has finished. Simplelayout blocks are excluded as well: cutting and pasting a block between pages, and the internal moves performed when a working copy is applied, are part of editing a page and are already covered by the page's own entries. Reordering blocks within a page never reaches this handler at all — that fires `IContainerModifiedEvent`. ## Entry Structure Each audit log entry stored in Elasticsearch has the following fields: | Field | ES Type | Description | |---|---|---| | `timestamp` | `date` | UTC ISO 8601 timestamp of the event. Note that Elasticsearch stores this at millisecond precision, so it is not a reliable sort key — use `seq`. | | `seq` | `long` | Monotonic per-index sequence number assigned under the chain lock. Defines the chain's true order and is covered by `hash`. | | `action` | `keyword` | One of the tracked action types listed above. | | `actor` | `keyword` | User ID of the person who performed the action. | | `actor_fullname` | `text` | Full name of the actor (for display). | | `content_uid` | `keyword` | UUID of the affected content object. | | `parent_uid` | `keyword` | UUID of the parent object. Populated only for block content so that per-object queries on a page also return changes to its blocks. | | `content_path` | `keyword` | Physical path of the object at the time of the event. | | `content_type` | `keyword` | Portal type of the content object (e.g. `ContentPage`, `Block`). | | `content_title` | `text` / `keyword` | Title of the object. Stored as both full-text and keyword sub-field. | | `details` | `object` (`enabled: false`) | Action-specific payload (changed fields, workflow info, etc.). Dynamic mapping is disabled to prevent Elasticsearch type conflicts when nested values alternate between types (e.g. `width` as string vs integer in layout JSON). | | `hash` | `keyword` | SHA-256 hash of this entry chained to the previous entry's hash. | | `prev_hash` | `keyword` | Hash of the immediately preceding entry (or `"genesis"` for the first entry in the chain). | ## Elasticsearch Index The audit log uses a dedicated ES index separate from the main Plone catalog index. The index name is derived from the Plone catalog index name with the suffix `-audit-log` appended (e.g. `abc123-audit-log`). The index is created with 1 shard, 0 replicas by default. It is created during initial site installation (`hooks.py`) and via the upgrade step for existing sites (`20260313100000_add_audit_log`). The write path calls `ensure_index` before storing an entry, so the index also gets created — with the explicit mapping, and with the hash chain reset — if a write arrives before either of those has run. This matters because Elasticsearch auto-creates an index on first write: without the check, a write during the window between deploying the code and running the upgrade step would create the index with *dynamic* mappings, turning `action` and `content_type` into `text` rather than `keyword`. That silently breaks the term filters and the `content_uid` aggregation behind previous values, and `create_index` could not repair it afterwards because it early-returns whenever the index exists. The check costs one `HEAD` request per stored entry (measured at roughly 1.6 ms, against ~10 ms to store an entry). It runs in the RQ worker, off the request path. It is deliberately not memoised: the default `rq.Worker` forks a work horse per job, so process-local state would not survive between jobs. Read paths do **not** check the index. If it is missing, queries return empty results and log a warning. ## Hash Chain Integrity Every audit log entry is linked to the previous entry through a cryptographic hash chain, similar to a blockchain. This makes it possible to detect if entries have been tampered with or deleted after the fact. **How it works:** 1. When a new entry is about to be stored, the worker acquires a Redis distributed lock (`audit_chain_lock_`, 5-second timeout) to serialize writes. 2. The previous hash and the last sequence number are read from the Redis keys `audit_chain_latest_` and `audit_chain_seq_`. If either is missing (Redis restart, eviction), both are recovered from Elasticsearch by querying the entry with the highest `seq`. For the very first entry, the sentinel value `"genesis"` and sequence 0 are used. 3. The entry's `seq` is set to the last sequence number plus one, and its `prev_hash` to the previous hash. 4. A SHA-256 hash is computed over the canonical JSON representation of the entry (keys sorted, concatenated with the previous hash). Because `seq` is assigned before this step, it is covered by the hash, so reordering entries is itself tamper-evident. 5. The resulting hash is stored in the entry's `hash` field and written to Elasticsearch. 6. The two Redis keys are updated to the new hash and sequence number. All Redis keys are namespaced by index name. A Zope instance can host several Plone sites, each with its own audit index, and they share one Redis database — global keys would interleave the sites' chains and leave every one of them unverifiable. **Why a sequence number rather than the timestamp:** Entries are timestamped when the event fires but chained when the transaction commits, and those two orders diverge whenever requests overlap: a slow request that stamped first can commit after a fast one that stamped later. Verifying in timestamp order would then report chain breaks on a perfectly intact chain. Elasticsearch also truncates the `date` type to milliseconds, so entries from a single request are frequently exact ties with no deterministic tiebreak. `seq`, assigned under the same lock that builds the chain, is by construction the chain's true order. **Verification:** The `verify_chain` function walks all entries from Elasticsearch in `seq` order and re-computes each hash. It detects: - Missing hash fields on entries - Chain breaks (an entry's `prev_hash` does not match the preceding entry's `hash`) - Hash mismatches (recomputed hash differs from stored hash, indicating the entry was modified after storage) Verification starts from the hash recorded in the index `_meta` under `chain_start` (see [Retention and Flush](#retention-and-flush)), falling back to `"genesis"` for an index that has never been flushed. Paging uses `search_after` on `seq` rather than `from`/`size`, which Elasticsearch caps at `index.max_result_window` (10 000 by default). An audit log outgrows that within its retention window on any busy site, and from/size paging would fail verification outright at that point. No point-in-time reader is needed: on a unique, increasing sort key, entries appended during the walk are simply picked up, never skipped or repeated. Verification can be triggered from the global audit log view (Manager only). **Resetting the chain:** When a fresh index is created (site installation, upgrade step, or manual creation from the admin view), the chain state in Redis is reset so the next entry starts a new chain from `"genesis"`. The reset only happens when the index does not yet exist, preventing accidental chain breaks on restarts. ## Recreating the Index The global audit log view offers Managers a **Recreate index** action that drops the index and creates it again with the current mapping, resetting the chain. It exists because `create_index` early-returns on an existing index, so an index carrying the wrong mapping cannot be repaired in place — for example one Elasticsearch auto-created before `ensure_index` was in place, or one predating a mapping change such as the addition of `seq`. The action is **destructive**: every stored entry is deleted, and the hash chain restarts from `"genesis"`. It therefore requires an explicit confirmation checkbox (enforced server-side, not only in the browser), is POST-only, and verifies the CSRF authenticator. The "Create Index" button shown when the index is missing is a different, non-destructive action; it only creates what is not there. ## Configuration The audit log is configured through three Plone registry records under the `wcs.backend.auditlog` prefix: `wcs.backend.auditlog.capture_enabled` (Bool, default: `True`) Controls whether content changes are recorded in the audit log. When disabled, no new entries are created but existing entries are preserved. Managers can toggle this from the global audit log view via the "Enable/Disable capturing" button. `wcs.backend.auditlog.enabled` (Bool, default: `False`) Controls visibility for non-Manager roles. When disabled, only Managers can access the audit log views and actions. When enabled, Site Administrators gain access to the global audit log and Editors gain access to per-object audit logs. See [Access Control](#access-control). `wcs.backend.auditlog.retention_days` (Int, default: `30`) Number of days to retain audit log entries. Used by the `flush-audit-log` console script. Entries older than this threshold are deleted during a flush run. The `capture_enabled` setting controls event *recording*. The `enabled` setting controls who can *view* the log and whether audit log actions appear in the UI for non-Manager roles. ## Access Control Access is governed by a combination of the `enabled` registry setting and Plone permissions: | Role | Scope | Access Rule | |---|---|---| | Manager | Global + per-object | Always has access, regardless of the `enabled` setting. | | Site Administrator | Global | Requires `enabled = True`. Uses the custom permission `wcs.backend Audit Log: View Global`. | | Editor | Per-object | Requires `enabled = True`. Uses the standard `Modify portal content` permission. | The custom permission `wcs.backend Audit Log: View Global` is granted to Manager and Site Administrator roles via `rolemap.xml`. The "Audit Log" actions in the object menu and user menu use an `available_expr` that checks the `enabled` registry record. The actions are only visible when enabled for editors or when the user is a Manager. Index creation, chain verification, and the capture toggle are restricted to Manager (`cmf.ManagePortal`). ## REST API The audit log exposes a `@audit-log` endpoint for programmatic access. **Global audit log** — query all entries across the site: ```javascript const response = await fetch('/Plone/@audit-log?b_start=0&b_size=25', { headers: { 'Accept': 'application/json', 'Authorization': 'Bearer ' } }); const data = await response.json(); console.log(data.items_total); console.log(data.items); ``` **Per-object audit log** — query entries for a specific content object, including changes to its child blocks: ```javascript const response = await fetch('/Plone/my-page/@audit-log', { headers: { 'Accept': 'application/json', 'Authorization': 'Bearer ' } }); const data = await response.json(); ``` **Query parameters:** | Parameter | Description | |---|---| | `b_start` | Pagination offset (default: 0). `b_start + b_size` must stay within Elasticsearch's result window of 10 000; beyond that the request is rejected with a 400 and the range has to be narrowed with `from` / `to`. | | `b_size` | Number of entries per page (default: 25, capped at 100). | | `action` | Filter by action type (e.g. `modified`, `created`). | | `actor` | Filter by user ID. | | `content_type` | Filter by portal type (global endpoint only). | | `from` | ISO date string for the start of a date range filter. | | `to` | ISO date string for the end of a date range filter. | **Response shape:** ```json { "@id": "http://localhost:8080/Plone/@audit-log", "items_total": 42, "items": [ { "timestamp": "2026-03-15T14:30:00+00:00", "action": "modified", "actor": "editor1", "actor_fullname": "Jane Editor", "content_uid": "abc-123", "parent_uid": "", "content_path": "/Plone/my-page", "content_type": "ContentPage", "content_title": "My Page", "details": { "changed_fields": { "title": {"new": "My Updated Page"} } }, "hash": "a1b2c3...", "prev_hash": "z9y8x7..." } ] } ``` The per-object endpoint automatically includes entries for child blocks by querying both `content_uid` and `parent_uid` matching the object's UUID. ## Working Copy Integration When a working copy (staging) operation runs, it triggers many internal content lifecycle events (add, copy, delete) that would pollute the audit log with noise. The staging system suppresses these by setting a request flag: ``` request._audit_staging_suppress = True ``` While this flag is set, the `on_content_added` and `on_content_removed` subscribers skip enqueuing entries. Once the staging operation completes, the flag is cleared and a single high-level event is fired: - `WorkingCopyCreatedEvent` — recorded as `working_copy_created` - `WorkingCopyAppliedEvent` — recorded as `working_copy_applied` - `WorkingCopyDiscardedEvent` — recorded as `working_copy_discarded` All three events fire against the **baseline** object, ensuring the audit trail for a piece of content remains consistent on its original UID regardless of working copy lifecycles. ## Login and Logout Tracking Sign-ins and sign-outs are captured plugin-agnostically by subscribing to the two standard PAS events, which every authentication path in the stack funnels through: - `Products.PluggableAuthService.interfaces.events.IUserLoggedInEvent` - `Products.PluggableAuthService.interfaces.events.IUserLoggedOutEvent` Coverage by authentication path: | Path | Login | Logout | |---|---|---| | Plone login form (`MembershipTool.loginUser`) | recorded | recorded | | `plone.restapi` `@login` / `@logout` (JWT) | recorded | recorded | | SAML (`wcs.samlauth`), incl. single logout | recorded | recorded | | OIDC (`pas.plugins.oidc`) | recorded | **not recorded** | | `wcs.adminauth` (CAS) | **not recorded** | **not recorded** | The two gaps are not fixable from a subscriber, because those code paths never fire the events: - `pas.plugins.oidc`'s logout views expire cookies and clear the session keyring directly, without calling `MembershipTool.logoutUser`. - `wcs.adminauth` establishes its session with a direct `updateCredentials()` call, bypassing `loginUser` entirely. Closing them requires changes in those packages, which are deliberately out of scope here. The actor is taken from `event.principal` rather than the current user: by the time the logout event fires, the security manager may already have been reset to Anonymous. Login and logout entries describe a session, not a content object, so `content_uid`, `content_path`, `content_type` and `content_title` are all empty. As a result they never match a per-object query and appear only in the global audit log. The `details` dict carries: **`ip`** Client IP, read from `CF-Connecting-IP` or `X-Forwarded-For` when present, otherwise `REMOTE_ADDR`. **`user_agent`** The raw `User-Agent` header. **`path`** The request path (e.g. `/acl_users/saml/acs`, `/@login`, `/login_form`). The path is stored verbatim rather than mapped to a plugin name — the events carry no indication of which PAS plugin fired them, so any such label would be guesswork. `pas.plugins.oidc` notifies both `UserInitialLoginInEvent` and `UserLoggedInEvent` on a user's first login, and the former subclasses the latter. A per-request flag therefore limits recording to one entry per action per request. Recording honours the `capture_enabled` registry setting like every other action. Failed login attempts are **not** recorded. ## Property Change Tracking Two content properties — `layout` (the selected view template) and `default_page` (the default child page) — are changed via OFS methods (`setLayout`, `setDefaultPage`) that do not fire `ObjectModifiedEvent`. To capture these changes, the audit log uses monkey patches registered via `collective.monkeypatcher`: - `BrowserDefaultMixin.setLayout` is wrapped to detect when the layout actually changes and enqueue a `property_changed` entry with old and new values. - `BrowserDefaultMixin.setDefaultPage` is wrapped similarly for default page changes. The patches use `preserveOriginal="true"` so the original methods are available as `_old_setLayout` and `_old_setDefaultPage`. ## Retention and Flush Old audit log entries can be purged with the `flush-audit-log` console script. It deletes all entries older than the configured `retention_days` using an Elasticsearch `delete_by_query`. **Usage:** ```bash bin/flush-audit-log /path/to/zope.conf ``` The script iterates over all Plone sites in the Zope instance and flushes each site's audit log index according to its own `retention_days` registry setting. This should be scheduled as a periodic cron job. Example: ``` 0 3 * * * /path/to/bin/flush-audit-log /path/to/zope.conf ``` Flushing does **not** invalidate chain verification. After deleting old entries, the flush records the oldest surviving entry's `prev_hash` in the index `_meta` under `chain_start`, and `verify_chain` seeds its walk from that value instead of `"genesis"`. A flushed index therefore still verifies clean, and every surviving entry remains individually tamper-evident (each entry's hash is recomputed against its own `prev_hash`, independent of whether its predecessor still exists). If the flush empties the index entirely, `chain_start` is set to the chain head at flush time, so the next entry written — which chains onto that head via Redis — still verifies. Deleting entries at the boundary is still detected: `chain_start` pins the expected predecessor of the oldest surviving entry, so removing that entry leaves the next one's `prev_hash` inconsistent with the recorded value. The limitation is that each flush run **re-baselines** `chain_start` from the index as it finds it. Boundary tampering that happened before a flush run is therefore absorbed into the new baseline and no longer reported. Detection at the boundary only holds until the next flush; deletions elsewhere in the retained range are always detected as chain breaks. If deletion detection over long periods matters, set a sufficiently high `retention_days` value. ## Previous Values for Modified Entries When displaying `modified` entries, the audit log view shows both old and new values for changed fields. The entry only stores the new value at the time of the event, so the previous value is reconstructed from the entry that precedes it: `query_previous_values` returns a mapping of each entry's `seq` to its predecessor's `changed_fields`, in a single ES request. The lookup is **per entry**, not per object. Sharing one "previous" across every entry of an object compares all but the newest against the wrong value -- the second-newest ends up compared against itself, suppressing every row and rendering an empty details modal. Entries are chained by `seq` rather than `timestamp`, for the same reason `verify_chain` walks by `seq`: concurrent requests invert the two. A working copy object is a clone, so it shares the chain of the object it was cloned from. That is what lets its *first* edit diff against the value the original had — without it a working copy is a brand new object with no history, and its first edit shows an empty previous value. The chain is keyed by `details["baseline_uid"]`, the UID of the object's own counterpart, which staging stamps onto every cloned object. It is deliberately *not* keyed by `parent_uid`: that points at the baseline page for everything inside a working copy, which would put the page and all of its blocks in one chain, so a block's "previous" would be whatever unrelated object was edited before it. A block added inside a working copy has no counterpart and chains onto its own history instead. The chain is deliberately asymmetric: a baseline entry only ever looks back at other baseline entries. A **discarded** working copy's values never reached the live object, so they must not become the old value of a later baseline edit. `PREVIOUS_LOOKBACK` bounds how far back the query reaches. A predecessor beyond that window leaves the entry without an old value, which renders as empty rather than as a wrong comparison. `created` entries record the values the content was created with, so they take part in the chain and the first edit after creation diffs against them. Only fields that actually hold something are recorded; empty defaults are skipped, which keeps the entry small and the details readable. Nothing precedes a create, so its own old values are empty. ## Excluded Content Types The following content types are excluded from audit log tracking because they are updated automatically, not by user actions: - **IAPIDataFetcher** — content that fetches data from external APIs. - **IRBSDataFetcher** — extends IAPIDataFetcher for RBS-specific data. Since `IRBSDataFetcher` extends `IAPIDataFetcher`, a single check on `IAPIDataFetcher.providedBy(obj)` covers both types. The check is applied in `on_content_added` and `on_content_modified`. ## Diff View for Modified Entries When viewing modified entries in the audit log detail modal, the system shows a unified diff (`difflib.unified_diff`) for fields with long or multiline values (HTML, JSON, layout fields). A "Show full values" collapse toggle reveals the complete old and new values side by side. Short single-line changes display old and new values directly without a diff. Dict values are pretty-printed as indented JSON for readability. ## Block Parent Tracking When a block (content marked with `IBlockMarker`) is modified, the audit log entry includes the `parent_uid` of the block's parent container. This enables the per-object audit log to show all changes to a page and its blocks in a single timeline. When querying by `object_uid`, the storage layer matches both `content_uid` and `parent_uid`, so a query for a page's UUID returns entries for the page itself plus all its blocks. ## Upgrade Path For new installations, the audit log index is created automatically during site setup (`hooks.py:create_audit_log_index`). For existing sites, the upgrade step `20260313100000_add_audit_log`: 1. Installs the upgrade profile which registers the `IAuditLogSettings` registry records and the `wcs.backend Audit Log: View Global` permission (granted to Manager and Site Administrator). 2. Registers the "Audit Log" action in both the object actions and user actions menus. 3. Creates the Elasticsearch audit log index, resetting the hash chain in Redis if it had to be created. The reset is conditional so that re-running the step against a populated index does not orphan the existing entries by restarting their chain from `genesis`.