Version: v2

AI Assistant

The docs site ships an AI-powered chat assistant — the CCG AI Assistant — that answers questions about Convenient Checkout using a Retrieval-Augmented Generation (RAG) pipeline built on LangChain. It's separate from AI-Powered Doc Search (the search-bar feature): the Assistant lives in the sidebar chat panel, remembers conversation context across follow-up questions, streams its answer token-by-token, and can generate example API requests grounded in the actual OpenAPI schema.

This page covers the complete implementation: architecture, components, retrieval strategy, conversation memory, and operations.

In this series:

AI Assistant — Overview & Architecture (this page)
AI Assistant — Prompt Engineering
AI Assistant — Improving the Assistant
AI Assistant — Indexing Internals

Two repositories

Unlike the docs site itself, the Assistant's backend is not part of this repo — it's a separate service:

Repo	Role
`wallet-ConvenientCheckout` (this repo)	UI only: `ChatBot.js` (chat widget) + `walletAIService.js` (API client)
`wallet-ai-service`	The actual RAG backend — FastAPI + LangChain, deployed as its own Kubernetes service

Nothing about the Assistant's retrieval, prompting, or generation logic lives in this repo. This repo only renders the chat UI and calls the backend's HTTP API.

Architecture

Browser (ChatBot.js)
  └─ walletAIService.askAssistantStream(query, {onToken, onDone, onError})
       └─ POST /api/ccg/chat/stream  { query, conversation_id, mode? }   (Server-Sent Events)
            └─ nginx (/api/ proxy → ai-service)
                  └─ wallet-ai-service (FastAPI, in-cluster)
                        ├─ Auth: Entra ID bearer token (fails closed if unset)
                        ├─ Rate limit: per-IP, shared with /api/ask
                        └─ CCGConversationalRagChain (LangChain)
                              ├─ 1. Condense — rewrite follow-up questions to standalone queries
                              ├─ 2. Retrieve — Azure AI Search, two-pass (spec docs + full corpus)
                              │        └─ wallet-docs-index-v3
                              ├─ 3. (optional) Generate example request — see below
                              ├─ 4. Generate answer — UAIS Gateway (gpt-5.6-terra), streamed
                              └─ 5. Persist turn to per-session conversation history

The backend is a standalone Kubernetes Deployment (ai-service) in the same namespace/cluster as the docs site. Nginx proxies /api/ browser requests to it — no direct browser-to-backend calls.

Request flow, step by step

ChatBot.js calls askAssistantStream(query, callbacks) in walletAIService.js.
The service POSTs to /api/ccg/chat/stream:
```
{ "query": "...", "conversation_id": "<per-tab session id>", "mode": "integration" }
```
(mode is optional — see Mode: Integration vs. Business.)
The backend validates the Entra ID bearer token (attached automatically via MSAL — see getAccessToken() in walletAIService.js), scopes the session by the authenticated subject, and checks the per-IP rate limit.
If there's prior conversation history, a lightweight LLM call rewrites the question into a standalone query (e.g. "what about for wallet mode?" → "what payment modes are available for wallet sessions?").
Azure AI Search is queried — two passes run in parallel for API-shaped questions (a spec-filtered pass + a full-corpus pass), merged and deduplicated.
If nothing relevant is found, the Assistant returns the mandated refusal — "I could not find that information in the CCG Microsite documentation." — without ever calling the LLM for an answer. No hallucination risk on a true "not found."
If the question resolves to a real API operation (see Example Request Generation), an example curl request is generated and attached to the context.
The full context (retrieved docs + citations + example request, if any) is sent to the UAIS Gateway (gpt-5.6-terra) along with the conversation history, and the answer streams back token-by-token.
The turn is persisted to that session's conversation history for future follow-ups.

Components

1. `src/components/ChatBot/walletAIService.js` — Browser AI service

Export	Description
`askAssistantStream(query, {onToken, onDone, onError})`	Used by `ChatBot.js`. Streams the answer via SSE — calls `onToken` per chunk, `onDone` once with citations, `onError` on failure.
`askAssistant(query)`	Non-streaming fallback — same backend, waits for the full answer. Not used by the current UI.
`askExplicitClient(query, config?)`	Direct, non-streaming POST to `/api/ccg/chat`.
`indexDocs(payload?)`	Triggers re-indexing via `POST /api/index` (admin-key gated on the backend).

Why SSE via fetch, not EventSource: the browser's native EventSource API can't send a POST body or a custom Authorization header — both are required here (the query, and the MSAL bearer token). askAssistantStream uses fetch + a manual ReadableStream reader, parsing data: {...} lines itself.

Session management: a UUID is generated per browser tab on first use and stored in sessionStorage as conversation_id. The backend uses it to key per-session conversation memory (further scoped server-side by the authenticated user — see Session isolation).

2. `src/components/ChatBot/ChatBot.js` — Chat widget

MarkdownContent — renders headings, bold, inline code, fenced code blocks (with CodeBlock's copy-to-clipboard button), bullet/numbered lists, tables, and the :::version-compare collapsible block.
Streaming placeholder bubble — sendQuery adds an empty assistant message immediately; onToken appends to it as chunks arrive. A bouncing-dots loader shows inside that bubble only while it's still empty (not a separate loading bubble).
RelatedDocs — groups citation links by URL pattern into labeled groups (via chatbotDocGroups.js).
Prompt suggestions — chips driven by prompt-suggestions.json, no code changes needed to update them.

Auth

Every call to /api/ccg/chat and /api/ccg/chat/stream requires a valid Entra ID bearer token — walletAIService.js acquires one silently via MSAL (same SSO session that gates developers.healthsafepay.com) and attaches it automatically.

The backend fails closed by default (AUTH_ENABLED=true): if the tenant/audience aren't configured for an environment, requests 503 loudly rather than silently falling back to anonymous access. This is deliberate — there's no NetworkPolicy restricting which pods in the cluster can reach the service, so network isolation alone isn't a substitute for verifying the caller's identity.

Session isolation

The raw conversation_id a browser tab generates is not used directly as the conversation-history key. The backend scopes it by the authenticated subject (oid, falling back to preferred_username) before touching the history store:

scoped_key = f"{subject}:{conversation_id}"

Without this, a caller who learned or guessed another tab's conversation_id could read or clear that conversation. Under AUTH_ENABLED=false (local dev only), every caller is "anonymous", so this degrades to a single shared namespace — a documented limitation of disabled auth, not a bug.

Conversation memory

Each session's history is an in-memory, per-process store (CCGSessionHistoryStore) — TTL-evicted, capped at a configurable max_turns. Two separate limits matter:

Setting	Purpose
`max_turns` (storage cap)	How many turns are retained for a session before the oldest are evicted.
`window_size` (prompt cap)	How many of those retained turns are actually sent to the LLM per request — bounds token usage/latency regardless of how long a conversation runs.

Concurrency: a per-session asyncio.Lock is held for the entire turn (condense → retrieve → generate → persist) — not just the history lookup. Two concurrent requests for the same session are serialized so they can't read the same starting history and then append out of order; requests for different sessions still run fully in parallel.

Streaming (astream_answer) and non-streaming (ainvoke) calls share the same lock and the same underlying history store — they're interchangeable per-request, not two separate conversation tracks.

Mode: Integration vs. Business

ChatRequest.mode ("integration" | "business" | None) lets the caller explicitly decide response style — it is never inferred from how the question happens to be phrased. This matters because keyword-based inference breaks down easily: a business question that happens to contain the word "curl" would get technical treatment, and a technical question phrased in plain language ("how do I issue a credit?") would get skipped.

Mode	Behavior
`"integration"`	Technical framing. If the question resolves to a real API operation, an example request is always attached — even without "curl"/"example" in the wording.
`"business"`	Prose-only framing. An example request is never attached, even if the question mentions "curl" or a generated example is available in context.
(omitted)	Today's default: decided per-question by a keyword heuristic (does the question mention "curl", "example", "payload", etc.).

A Business / Integration toggle in the chat header sets this explicitly on every request (persisted per-tab in sessionStorage) — it defaults to "business".

Suggested prompts also follow this toggle. Each group in prompt-suggestions.json can carry a modes array (e.g. ["business"]); ChatBot.js filters the suggestion chips shown to just the groups matching the current mode before rendering, so switching Business ↔ Integration changes which questions are suggested, not only how the answer to a given question is framed. A group with no modes field always shows. Today: Business prompts (plain-language) only appear in Business mode; Integration and API (including its Errors sub-group) only appear in Integration mode.

Search: always hybrid (keyword + vector)

Every chat request explicitly targets wallet-docs-vector-index-v1 (walletAIService.js's VECTOR_INDEX_NAME constant) — the hybrid (keyword + vector-embedding) index, more forgiving of questions that share little wording with the doc that actually answers them (see Indexing Internals — Vector (hybrid) search).

There used to be a Basic / Advanced search-mode toggle here, letting a user pick between the old keyword-only index and this one — a deliberate, temporary validation control (see wallet-ai-service's docs/ccg_ai_assistant/12-vector-search-plan.md) so the new index could be validated against real questions without a code change per test. That validation is done; the toggle has been removed, and hybrid search is now simply how this Assistant always searches — not a user-facing choice.

index_name is still sent explicitly (not omitted) on every request, because the server's own default index (AZURE_SEARCH_INDEX) has not itself been cut over to wallet-docs-vector-index-v1 — see wallet-ai-service's src/config/index.py.

Example Request Generation

When a question resolves to a specific API operation, the Assistant can attach a ready-to-use example curl command — grounded in the actual OpenAPI schema, not whatever (possibly empty) example happens to be baked into the indexed documentation page.

Finding the right operation — carefully

Retrieved citation URLs already encode an operationId (e.g. /api-reference-v2/#tag/Merchant/operation/createSession). Picking the right one, when several API operations are retrieved for one question, is done in three steps, each escalating only when the previous one is inconclusive:

Title word-overlap. Score each candidate operation by how many words its title shares with the question, normalized by title length. Cheap, precise, and correct for direct matches ("create a session" → createSession, not createChildCheckoutSession).
Safe single-candidate fallback. If no title scores above zero — a paraphrase the title-matching can't see, e.g. "issue credit" vs. "Create Refund" — and there's only one distinct operation among the retrieved docs, use it. There's no competing candidate to mispick.
LLM disambiguation, bounded to real candidates. If there are multiple distinct operations and no title winner, ask the LLM to choose from that closed list (id, title, summary) — or say NONE. The model can never introduce an operation it wasn't handed; a response that isn't exactly one of the given ids is treated as no match, not trusted.

This exists because raw search relevance ranking is a relevance signal, not a disambiguation signal — the correct operation for "issue credit" (createRefund) has ranked as low as 7th out of 10 retrieved documents in practice.

Building the example body

Once an operation is identified, the request body is built two ways, in order:

LLM-generated, schema-validated. The LLM is given the operation's fully-dereferenced JSON Schema and the use case, and asked to produce a JSON body using only field names and enum values that literally exist in that schema. The result is validated against the same schema (jsonschema, with additionalProperties: false injected — the spec itself doesn't set it, and without it any hallucinated field would silently pass validation) before being trusted.
Deterministic fallback. If the LLM call fails, returns invalid JSON, or the body doesn't validate, a purely schema-driven synthesizer builds a body from example/default/enum values — zero hallucination risk, but structurally limited (can't represent a use case that needs a materially different shape than the first few schema properties).

The generated block is injected into the LLM's context labeled "Generated Example Request," and the system prompt requires it be reproduced verbatim in the final answer — the model narrates around it but never edits its contents.

Where the spec comes from

The OpenAPI schema used for this is the Redocusaurus-bundled v2 spec (build/redocusaurus/plugin-redoc-1.yaml — where every cross-file $ref is already merged into same-file pointers), fetched by the backend from a fixed-key document in Azure AI Search (not a ranked search — a direct key lookup, since the generator needs the spec deterministically). See Indexing Internals for how it gets there.

Retrieval — Azure AI Search

The retriever (AzureAISearchLangChainRetriever) queries the same wallet-docs-index-v3 used by AI-Powered Doc Search (both fall back to the same AZURE_SEARCH_INDEX setting — see wallet-ai-service's src/app/ask/service.py). For API-shaped questions, it runs a two-pass search in parallel:

Pass 1 — filtered to category eq 'open-api-spec', prioritizing spec/operation documents.
Pass 2 — the full corpus, no filter.

Results are merged and deduplicated by file_path, spec results first. See Indexing Internals for how documents get into the index in the first place.

Generation — UAIS Gateway

Unlike a direct Azure OpenAI deployment, the Assistant calls gpt-5.6-terra through UHG's UAIS LLM Gateway — a shared, managed endpoint, not a dedicated deployment this team owns.

Property	Value
Model	`gpt-5.6-terra`
Deployment	`gpt-5.6-terra_2026-07-09`
Gateway	`https://api.uhg.com/api/cloud/api-management/ai-gateway-reasoning/1.0`
Auth	OAuth 2.0 client-credentials (not an `api-key` header)
Quota	20K TPM

Auth flow: the backend's UAISTokenProvider fetches an OAuth token via client-credentials grant, caches it, and refreshes proactively before expiry. On a 401 from the gateway, it force-refreshes once and retries a single time (not a retry loop).

Reasoning-model quirks the backend already accounts for:

Uses max_completion_tokens, not the deprecated max_tokens.
Only supports the default temperature (1) — the backend omits the field entirely rather than sending its usual non-default value.
The gateway requires a projectId header (quota/billing attribution) on every call.

Streaming: the gateway supports real Server-Sent token streaming (stream: true in the request body) — the backend's UAISGatewayChatModel._astream parses the SSE response and yields chunks as they arrive, which is what makes /api/ccg/chat/stream genuinely incremental rather than "wait for the whole answer, then fake a stream."

Rate Limiting & Caching

Rate limiting is per-client-IP, shared with /api/ask (same singleton limiter instance) — default 5 requests/minute. Client IP is resolved via src/core/http.py::client_ip, which only honors X-Forwarded-For when the backend is explicitly configured to trust it (TRUST_PROXY_HEADERS) — otherwise it uses the raw TCP peer address, since a direct caller can set XFF to anything it wants.

No response cache currently exists for /api/ccg/chat//chat/stream — every request re-runs retrieval and generation. (/api/ask has an in-process LRU cache; the CCG chat endpoints don't share or use it, since conversation-aware responses aren't safely cacheable by query text alone.)

Deployment

The backend is deployed from the wallet-ai-service repo via its own Helm chart — see that repo for the full chart. Key environment variables relevant to this Assistant:

Variable	Purpose
`AUTH_ENABLED`	Fails closed (`true`) by default — see Auth.
`AZURE_AD_TENANT_ID` / `AZURE_AD_AUDIENCE`	Entra ID tenant/audience for bearer-token verification.
`UAIS_CLIENT_ID` / `UAIS_CLIENT_SECRET`	OAuth2 client-credentials for the UAIS Gateway.
`UAIS_PROJECT_ID`	Sent as the `projectId` header on every gateway call.
`CCG_OPENAPI_V2_SPEC_DOC_ID`	Azure Search document key for the bundled OpenAPI spec (example-request generation).
`AZURE_SEARCH_ENDPOINT` / `AZURE_SEARCH_API_KEY` / `AZURE_SEARCH_INDEX`	Same setting/value used by Doc Search — today, `wallet-docs-index-v3`.
`CCG_CONVERSATION_MAX_TURNS` / `CCG_CONVERSATION_WINDOW_SIZE` / `CCG_CONVERSATION_TTL_SECONDS`	Conversation memory tuning — see Conversation memory.

nginx proxies /api/ to the backend the same way it does for /api/ask and /api/index — no separate proxy configuration needed for the CCG endpoints.

Troubleshooting

Symptom	Cause	Fix
`{"detail":"Authentication is not configured on this server."}`	`AUTH_ENABLED=true` but tenant/audience not set for this environment	Set `AZURE_AD_TENANT_ID`/`AZURE_AD_AUDIENCE`, or (local dev only) `AUTH_ENABLED=false` in an uncommitted `.env`
Chat panel shows "unavailable"	`/api/ccg/chat/stream` returning non-2xx or a stream `error` event	Check `ai-service` pod logs
403 from Azure Search	IP not on the Search resource's network allowlist, or wrong key	Check Azure Portal → Search resource → Networking
403 "UAIS gateway... verify deploymentName, modelName"	Wrong request shape or unapproved model/deployment for the configured `UAIS_PROJECT_ID`	Confirm `UAIS_MODEL_NAME`/`UAIS_DEPLOYMENT` match what's actually approved for the project
Answer has an empty `-d '{}'` curl body	Retrieved doc's own baked-in example is empty and the operation-matching step found nothing to generate from	Check backend logs for `Example-curl generation failed` or a skipped disambiguation
Curl attached to a business-sounding question	`mode` wasn't sent, or was sent as `"business"` incorrectly	Confirm the request payload; `mode="business"` always suppresses example generation

Two repositories​

Architecture​

Request flow, step by step​

Components​

1. src/components/ChatBot/walletAIService.js — Browser AI service​

2. src/components/ChatBot/ChatBot.js — Chat widget​

Auth​

Session isolation​

Conversation memory​

Mode: Integration vs. Business​

Search: always hybrid (keyword + vector)​

Example Request Generation​

Finding the right operation — carefully​

Building the example body​

Where the spec comes from​

Retrieval — Azure AI Search​

Generation — UAIS Gateway​

Rate Limiting & Caching​

Deployment​

Troubleshooting​