Improving the Assistant
This page answers the most common question about the AI Assistant: "Can we train this to know more / behave better?"
Short answer: you can't fine-tune gpt-5.6-terra yourself — it's accessed through UHG's shared UAIS Gateway, not a dedicated deployment this team owns. But there are several progressively more effective ways to make the Assistant better without touching the model at all, and the cheapest ones require no wallet-ai-service code changes.
Option 1: Improve the Index (Free, Highest ROI)
What it does: feeds the model more complete, more accurate documentation to answer from.
The Assistant retrieves from wallet-docs-index-v3 (today's AZURE_SEARCH_INDEX value) and passes it as context — if the retrieved content is missing, stale, or thin, the model has nothing to work with regardless of how good it is.
| Symptom | Root cause | Fix |
|---|---|---|
| "I could not find that information..." for a documented feature | Page missing from the index, or excluded as noise | Confirm the page isn't under a _-prefixed folder (drafts are intentionally excluded) or release-notes (also excluded), then re-index |
| Answer omits fields that exist in the docs | Content wasn't indexed at that path, or got truncated | Check the page has an # H1 or front-matter title: — files without either used to be silently dropped entirely (fixed), but still worth checking |
Empty -d '{}' curl body for an endpoint | The indexed doc's own baked-in example is empty, and operation-matching found nothing to generate from | See Example Request Generation — this is a schema/generation issue, not a doc-content issue |
| Wrong v1/v2 answer | Both versions indexed under the same terms | Tag content explicitly (category) so retrieval and citations distinguish them |
Re-indexing
Indexing logic lives in wallet-ai-service's src/app/shared/indexer/, triggered via the admin-key-gated POST /api/index on that backend — see Indexing Internals for what actually gets parsed and how.
curl -X POST https://<ai-service-host>/api/index \
-H "Content-Type: application/json" \
-H "X-Admin-Key: <admin key>" \
-d '{
"index_data": {
"docs": [{ "directory": "docs", "category": "docs" }],
"yaml": [{ "directory": "openapi/v2/apispec.yaml", "category": "open-api-spec-v2" }]
}
}'
There is currently no automatic re-index on merge — this repo doesn't yet have a CI step that calls /api/index after a docs change lands on main. Until that exists, re-indexing is a manual step after meaningful documentation changes.
Option 2: Improve the System Prompt (Free, Fast Iteration)
What it does: changes how the model interprets retrieved context and formats its answer.
See Prompt Engineering for the full prompt set. In short, editing wallet-ai-service's src/app/ccg_ai_assistant/prompts.py lets you:
- Tighten hallucination controls if the model starts inferring undocumented fields.
- Adjust the mode addenda if integration/business framing needs to shift.
- Change response formatting rules.
Prompt changes take effect on the next wallet-ai-service deploy — no re-indexing needed, and nothing in this repo changes.
Option 3: Fine-Tuning — Not Available Here
Fine-tuning trains a copy of a base model on your own example conversations, producing a deployment with baked-in domain knowledge. Azure OpenAI supports this for direct Azure OpenAI deployments — but gpt-5.6-terra isn't one. It's served through UHG's UAIS Gateway, a shared, managed, multi-tenant reasoning endpoint (see Generation — UAIS Gateway). There is no mechanism, today, for this team to fine-tune the model behind that gateway.
If model behavior genuinely can't be fixed by better retrieval or a better prompt, the actual lever available is asking whoever owns the UAIS Gateway/model catalog whether a different approved model better fits the use case — not fine-tuning the current one.
Option 4: Expanding the Knowledge Base
What it does: adds new content types to the index, not just markdown pages and OpenAPI specs.
The indexer already supports three source types (markdown.py, yaml.py, json.py in wallet-ai-service), plus an inline-documents path for anything pre-formatted (index_inline_documents — see Indexing Internals). Adding a genuinely new source type (e.g. Confluence pages, GitHub issues) means writing a new parser module in wallet-ai-service's src/app/shared/indexer/ following the existing pattern (build a DataFrame with id/title/content/file_path/category, reuse the shared upload pipeline) — this is backend work, not something this repo can do alone.
Option 5: Improve Example Request Generation
What it does: improves the quality of generated curl examples, independent of the system prompt or the index.
Because example generation is grounded directly in the OpenAPI schema (see Example Request Generation), the highest-leverage improvements are spec changes, not prompt changes:
| Symptom | Fix |
|---|---|
| Generated example uses an unhelpful placeholder value | Add an example: to that field in the OpenAPI schema — both the LLM-grounded and deterministic paths prefer explicit example/default values over guessing |
| Generated example picks the wrong enum member for the described use case | Ensure the enum's own description documents what each member means (the intent-matching logic scores against that text) |
| Wrong operation attached entirely | Check whether the operation's summary/title is too generic to disambiguate from a similarly-named one — see Finding the right operation |
Summary: which option for which problem?
| Goal | Best option | Effort |
|---|---|---|
| Assistant doesn't know about a feature | Add/improve the doc → re-index | Low |
| Assistant invents fields or endpoints | Tighten the system prompt | Low |
| Assistant uses wrong tone for the audience | Adjust mode addenda | Low |
| Assistant gives stale answers | Re-index after doc changes | Low |
| Generated curl has a poor example value | Add example:/enum descriptions to the OpenAPI spec | Low |
| Assistant needs a new content type indexed | Write a new indexer parser (backend work) | Medium |
| Model itself needs to behave fundamentally differently | Not available via fine-tuning — raise with the model/gateway owner | High / out of this team's control |