Case study
xanderGPT
A self-hosted ChatGPT-style app powered by Qwen3.5 through Ollama, with streamed reasoning, persistent conversations, context compaction, and web-aware tool orchestration.
Technology
- TypeScript
- React
- Node.js
- Express
- Tailwind
- Prisma
- MySQL
- Ollama
- Role
- Sole developer
- Year
- 2026
- Status
- Local prototype
Context
Learning what sits behind an AI chat interface
xanderGPT did not begin with a problem I needed to solve. I wanted to understand how difficult it would be to build my own ChatGPT-style application and how the interface, model, conversation state, and external tools fit together.
I treated it as a local prototype rather than a production product, focusing on recreating the parts of an AI chat experience that users normally take for granted.
The system
A local model with persistent conversations
The React frontend communicates with an Express API that sends prompts to a local Qwen3.5 9B multimodal model through Ollama. I moved from Qwen3 8B to Qwen3.5 9B to add image understanding while retaining strong instruction following and configurable reasoning. The model remains practical to run on my RTX 3070, with some layers offloaded to system memory when required.
Users can attach images through the file picker, drag and drop, or clipboard paste. The frontend previews and resizes them to a maximum of 1024 pixels before the backend passes the image data to Ollama alongside the prompt.
Prisma and MySQL store accounts, conversations, messages, image attachments, per-chat preferences, reasoning traces, and hidden context summaries for long-running chats. The backend streams reasoning and final answers separately, supports cancellation and temporary chats, and creates automatic titles. Users can also search conversation titles and message contents, then jump directly to a highlighted match.
Architecture at a glance
How a message moves through xanderGPT
Each request moves through planning and context assembly before reaching the local model. The resulting reasoning, tool activity, sources, and answer are streamed to the interface, then persisted with the conversation.
Tool orchestration
One planner, several bounded capabilities
Before generating an answer, the backend uses one planner to decide whether the request needs web search, calculator use, extended reasoning, or a combination of them. It considers the request, recent history, deterministic freshness cues, the current runtime date, and any options explicitly enabled by the user.
Assistant, routing, tool, retrieval, compaction, and runtime instructions are assembled through dedicated prompt builders instead of being embedded throughout the orchestration services. Follow-up questions become self-contained searches, calculations use a bounded evaluator, and pasted URLs are fetched and reduced to readable content. Users can still force web search or reasoning for the next message while leaving both automatic by default.
Reasoning experience
Separating intermediate reasoning from the answer
Qwen3.5 can emit a reasoning stream separately from its final response. xanderGPT displays it in a compact panel that follows the latest token while the model is working, then collapses into a summary such as "Thought for 15 seconds" when the answer begins. The trace can be reopened and is persisted with saved conversations.
- Reasoning traces remain separate from the final message and are never fed into later conversation context.
- The planner skips reasoning for straightforward requests, while a one-message toggle lets users enable it for harder prompts.
The hardest reliability problem
Reliable web search is more than an API call
The first implementation passed Brave result titles and snippets directly to the model. Testing exposed the gap between having search results and having evidence: answers could be weakly grounded or stale, queries were sometimes poor, and the interface did not clearly connect claims to sources.
- 01
Search
Begin with a small result set based on the user's request and recent context.
- 02
Assess
Check whether the results are sufficient and rewrite the query once if needed.
- 03
Select
Prefer relevant primary and reputable sources while removing weak or duplicate pages.
- 04
Extract
Fetch selected pages, reduce them with Readability, and retain the strongest passages.
- 05
Answer
Give the model only that evidence and map its numbered citations to stored sources.
- Failed extraction falls back to a labelled snippet, while strict caps on searches, fetches, timeouts, page size, and retained passages keep the process finite.
Remaining limitations
More capable, still a local prototype
- Retrieval
- Readability can fail on dynamic, protected, or poorly structured pages. Passage and conversation search rely on lexical overlap, so related wording can still be missed.
- Latency, memory, and context
- Fetching, reasoning, image processing, and context compaction add latency, especially when the multimodal model is partially offloaded to system memory. Images are resized before inference to balance visual detail against memory use. Long reasoning traces share a finite context window with the prompt and final answer, so difficult requests can exhaust the available budget before the model produces a response. Context summaries extend longer chats but can still omit or misclassify details.
- Model reliability
- The local 9B model can produce malformed structured output or uneven reasoning, so deterministic constraints and safe fallbacks guard every model-controlled decision.
What I learned
Tool use depends on the quality of the handoff
The project showed me that an AI chat experience depends less on any single capability than on the handoffs between them. The difficult work is defining clear contracts between prompts, planners, tools, models, and the interface: deciding when fresh evidence or deeper reasoning is useful, keeping those operations bounded, and presenting the result clearly enough that users understand what happened and where it may still be weak.