CodeIssuesPull RequestsActionsSecurityInsights
✨ AI
More
Settings

feat(ai-provider): messages.stream — the adapter half that was missing #5627

MergedXSccantynz wants to mergefeat/ai-provider-streamingmainopened 19h ago2/2 tasks
ccantynzcommented 19h ago

Summary

PR B of the multi-provider build. AiClientLike implemented only messages.create, so AI_PROVIDER=openai crashed all four streaming features (personal-chat, repo-chat, spec-ai, streaming-review) with messages.stream is not a function.

  • One SSE pump drives the connection regardless of consumption style, serving both styles the features use: for await over Anthropic-shaped events (content_block_delta / message_delta with usage / message_stop) and .on("text") + .finalMessage().
  • stream_options.include_usage asks for token counts on the tail chunk so cost tracking sees real numbers; servers that ignore it degrade to 0, never NaN.
  • Failure is recorded before the iterator wakes — a truncated stream ends loudly, not cleanly. Torn SSE frames skip; frames split across reads reassemble.

With this merged, one env var (AI_PROVIDER=openai + AI_BASE_URL) points the entire platform — streaming features included — at OpenAI, Azure, OpenRouter, Groq, Together, Ollama, vLLM, or llama.cpp.

Test plan

  • 6 new tests: both consumption styles, finalMessage-only drain, non-2xx rejection in both styles, torn frame, split frame — pass
  • Existing ai-provider suite green (18 total), tsc --noEmit clean, bun build clean
ccantynzAI Reviewcommented 19h ago

AI Triage

(no summary)

Priority: medium Risk area: mixed

Suggested labels: (no label suggestions) Suggested reviewers: (no reviewer suggestions)

Suggestions only — nothing has been applied. The PR author stays in control.

gluecron[bot]🤖 botAI Reviewcommented 19h ago

AI review unavailable

The platform's AI balance is exhausted, so AI generation is temporarily unavailable. Nothing was lost. You can queue this as a repair for the internal agent from the repository's Health page, or try again once the balance is restored. The PR is otherwise unchanged.

Cross-repo impact

See what breaks downstream if this PR merges.

Analyze →
⮌ Merged

This pull request was merged into main.

c comment · e edit title · m merge · a approve · r request changes · ? shortcuts