# Chatbot

Conversational assistant built on the same vector retrieval substrate as
[RAG](rag.md), with multi-turn memory and a consent-gated hand-off to staff.

The chatbot is a **separate, independently-toggleable feature**. It reuses the
shared AI core (`wcs.backend.ai`: provider client, retrieval, grounding) and the
RAG chunks index, but ships its own REST surface, browser page and
configuration. Enabling the chatbot does not enable the RAG question surface and
vice versa.

## How it differs from RAG

| | RAG (`@rag-ask`) | Chatbot (`@chatbot-ask`) |
|---|---|---|
| Interaction | Single question / answer | Multi-turn conversation with memory |
| Follow-ups | Not supported | Each turn is condensed into a standalone query before retrieval |
| Hand-off | None | Consent-gated forwarding to staff (email + stored ticket) |
| Toggle | `RAG_ENABLED` | `CHATBOT_ENABLED` |

Both features generate answers only from retrieved, security-filtered context.
The chatbot additionally verifies answerability: after retrieval, a lightweight
LLM classification call decides whether the retrieved context actually answers
the question. The bot proactively offers to forward the question to staff both
when nothing relevant is found **and** when the retrieved context does not
answer it.

## Independent toggle

- `RAG_ENABLED` gates the RAG question/admin surface only.
- `CHATBOT_ENABLED` gates the chatbot surface only.
- The shared retrieval substrate (indexing subscribers, chunk read/write) is
  active whenever **either** feature is enabled, so a chatbot-only site indexes
  and retrieves content normally without the RAG surface being on.

## Per-content enablement

`CHATBOT_ENABLED=true` is the global master switch, but it does not turn the
chatbot on everywhere. The chatbot is additionally gated per content object by
an opt-in toggle, mirroring the per-book RAG toggle. A given content is
chatbot-enabled only when the global flag **and** its per-item toggle are both
on.

The toggle is an "Enable chatbot" checkbox contributed by the
`chatbot.configuration` behavior. The behavior ships **registered but not
attached to any content type** by the default profile. To use it:

1. A site admin enables the `chatbot.configuration` behavior on the desired
   content types in **Site Setup → Content Types**.
2. Editors then tick **Enable chatbot** on individual items of those types.

The `@@chatbot` browser page and the `@chatbot-ask` / `@chatbot-escalate` REST
endpoints only respond on content whose chatbot is enabled; elsewhere the page
returns 404 and the endpoints refuse. Editing the "Enable chatbot" field
requires the `Manage chatbot properties` permission (granted to Manager).

## Configuration

### Environment variables

```bash
# Feature flag
CHATBOT_ENABLED=true
```

The chatbot reuses the shared `EMBEDDING_*` and `LLM_*` provider settings and
the shared `RAG_TOP_K` / ranking configuration documented in [RAG](rag.md).

### Plone registry

| Record | Description |
|---|---|
| `wcs.backend.chatbot_system_prompt` | System prompt for grounded, multi-turn answers |
| `wcs.backend.chatbot_condense_prompt` | Prompt that rewrites a follow-up into a standalone query |
| `wcs.backend.chatbot_answerable_prompt` | Prompt for the JA/NEIN check that decides whether the retrieved context answers the question (drives the escalation offer) |
| `wcs.backend.chatbot_no_answer_message` | Message when no relevant context is found (offers escalation) |
| `wcs.backend.chatbot_error_message` | Message when a turn fails |
| `wcs.backend.chatbot_llm_temperature` | Answer temperature (default `0.2`, kept low for grounding) |
| `wcs.backend.chatbot_llm_max_tokens` | Maximum tokens per answer |
| `wcs.backend.chatbot_max_turns` | Conversation turns retained per session |
| `wcs.backend.chatbot_session_ttl` | Seconds a session stays alive (sliding expiry) |
| `wcs.backend.chatbot_turn_result_ttl` | Seconds a turn result is cached for polling |
| `wcs.backend.chatbot_top_k` | Chunks retrieved per turn (default `5`; own knob so history doesn't crowd the context window) |
| `wcs.backend.chatbot_max_message_chars` | Maximum length of a single user message (anti-abuse input cap) |
| `wcs.backend.chatbot_condense_temperature` | Temperature for the follow-up condensation call |
| `wcs.backend.chatbot_condense_max_tokens` | Maximum tokens for the condensed standalone query |
| `wcs.backend.chatbot_support_email` | Email of the staff member who receives escalations outside a book context (should match a Plone member) |
| `wcs.backend.chatbot_system_initiator` | Fallback identity used to create tickets for anonymous escalations (default `adminuser`) |
| `wcs.backend.chatbot_escalation_action` | Task action term for escalation tickets (default `chatbot-anfrage`) |
| `wcs.backend.chatbot_escalation_subject` | Subject line for escalation emails |
| `wcs.backend.chatbot_escalation_confirmation` | Confirmation message emailed to the visitor |

## Escalation routing

When a visitor consents to forwarding a conversation, a `Task` ticket with the
full transcript is stored in the existing task system — in the task container of
the staff member who handles it — and staff are notified:

- **Inside a book**: the ticket is owned by and assigned to the book owner
  (moderator), who is notified through the existing task-notification content
  rule, exactly like the "ask the book owner" flow.
- **Everywhere else**: the ticket is owned by and assigned to the Plone member
  whose email is configured in `chatbot_support_email`, who is then emailed
  directly. Configure this to a real staff member so the ticket lands in their
  task list. If the email does not match a member, an authenticated visitor's
  ticket goes to their own task container and anonymous visitors are notified by
  email only.

The visitor always receives a confirmation at the email they provided. Anonymous
visitors are supported. Each conversation yields at most one ticket.

## REST API

### Send a message (async)

By default a turn is processed asynchronously; the client polls for the result.
The first response mints a `conversation_id` that must be sent back on every
following turn to keep conversation memory.

```javascript
const response = await fetch('/Plone/@chatbot-ask', {
    method: 'POST',
    headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' },
    body: JSON.stringify({
        message: 'Wie sind die Öffnungszeiten?',
        path: '/plone/section'  // optional: restrict retrieval to a section
    })
});
const data = await response.json();
// data.status === 'pending', data.job_id, data.conversation_id
```

**Pending response:**

```json
{
  "@id": "http://localhost:8080/Plone/@chatbot-ask",
  "status": "pending",
  "job_id": "chatbot_turn_ab12…_0",
  "conversation_id": "ab12cd34ef56ab12cd34ef56ab12cd34",
  "turn_index": 0
}
```

**Poll for the result:**

```javascript
const poll = await fetch('/Plone/@chatbot-ask?job_id=' + data.job_id, {
    headers: { 'Accept': 'application/json' }
});
const result = await poll.json();
```

**Completed response:**

```json
{
  "@id": "http://localhost:8080/Plone/@chatbot-ask",
  "status": "completed",
  "conversation_id": "ab12cd34ef56ab12cd34ef56ab12cd34",
  "turn_index": 0,
  "answer": "Die Öffnungszeiten sind …",
  "sources": [
    {"title": "Kontakt", "path": "/kontakt", "portal_type": "Contact", "score": 0.92}
  ],
  "grounded": true,
  "escalation_offer": false
}
```

`answer` is plain text. `grounded` reflects whether the retrieved context
actually answers the question: it is `false` both when no relevant context was
found and when the found context does not answer the question. When `grounded`
is `false`, `escalation_offer` is `true` and the bot suggests forwarding the
question to staff.

### Continue the conversation

Send the `conversation_id` from the first response back on every subsequent
turn. The running history is condensed into a standalone query used **for
retrieval**, so follow-up questions work without repeating context. The answer
itself is then generated from the user's original question plus the
conversation history.

```javascript
await fetch('/Plone/@chatbot-ask', {
    method: 'POST',
    headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' },
    body: JSON.stringify({
        message: 'Und am Wochenende?',
        conversation_id: 'ab12cd34ef56ab12cd34ef56ab12cd34'
    })
});
```

Logged-in clients may add `"sync": true` to receive the answer directly instead
of polling.

### Forward to staff

After the visitor consents, forward the conversation. `consent` must be `true`.

```javascript
const response = await fetch('/Plone/@chatbot-escalate', {
    method: 'POST',
    headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' },
    body: JSON.stringify({
        conversation_id: 'ab12cd34ef56ab12cd34ef56ab12cd34',
        consent: true,
        contact_email: 'visitor@example.com',  // optional
        path: '/plone/section'                  // optional: enables book-owner routing
    })
});
const data = await response.json();
// data.status === 'forwarded' (or 'already_forwarded' on a repeat)
```

**Response:**

```json
{
  "@id": "http://localhost:8080/Plone/@chatbot-escalate",
  "status": "forwarded",
  "conversation_id": "ab12cd34ef56ab12cd34ef56ab12cd34"
}
```