Skip to content

fix: Litellm preserve streamed reasoning deltas in LiteLLM adapter#4952

Open
lorenzbaraldi wants to merge 2 commits intogoogle:mainfrom
lorenzbaraldi:fix/reasoning-accumulation
Open

fix: Litellm preserve streamed reasoning deltas in LiteLLM adapter#4952
lorenzbaraldi wants to merge 2 commits intogoogle:mainfrom
lorenzbaraldi:fix/reasoning-accumulation

Conversation

@lorenzbaraldi
Copy link

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

No existing issue linked for this fix.

Problem:
In LiteLlm message conversion, reasoning parts were combined with newline
injection:

reasoning_content = _NEW_LINE.join(text for text in reasoning_texts if text)

For providers that stream reasoning in delta fragments (for example, vLLM-style
reasoning chunks), this mutates the original stream by inserting extra
separators. The reconstructed reasoning can differ compared to provider
output.

Solution:
Preserve reasoning text exactly as streamed by concatenating fragments without
adding separators:

reasoning_content = "".join(text for text in reasoning_texts if text)

This avoids corruption of chunked reasoning while still preserving explicit
newlines already present in fragments.

Also added targeted regression tests to lock behavior:

  • test_content_to_message_param_preserves_chunked_reasoning_deltas
  • test_content_to_message_param_preserves_reasoning_newlines

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Summary of local pytest runs:

  1. python -m pytest tests/unittests/models/test_litellm.py -k "content_to_message_param_assistant_thought_and_content_message or preserves_chunked_reasoning_deltas or preserves_reasoning_newlines"
  • Result: 3 passed, 244 deselected
  1. python -m pytest tests/unittests/models/test_litellm.py -k "preserves_chunked_reasoning_deltas or preserves_reasoning_newlines"
  • Result: 2 passed, 245 deselected

Manual End-to-End (E2E) Tests:

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

Scope is intentionally minimal and low risk:

  • 1-line behavior change in reasoning-content reconstruction.
  • 2 regression tests added.
  • Anthropic thinking_blocks path is unchanged.

@adk-bot adk-bot added the models [Component] Issues related to model support label Mar 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models [Component] Issues related to model support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants