Skip to content

fix(gemini): allow thinking_config passthrough - #411

Draft
autodots-AHassan wants to merge 1 commit into
google:mainfrom
autodots-AHassan:fix/gemini-thinking-config-319
Draft

fix(gemini): allow thinking_config passthrough#411
autodots-AHassan wants to merge 1 commit into
google:mainfrom
autodots-AHassan:fix/gemini-thinking-config-319

Conversation

@autodots-AHassan

@autodots-AHassan autodots-AHassan commented Mar 8, 2026

Copy link
Copy Markdown

Description

Add thinking_config to the Gemini provider allowlist so language_model_params={"thinking_config": ...} survives GeminiLanguageModel.__init__ filtering and is forwarded to the Gemini API. This also updates the provider docstring and adds regression coverage for both constructor-time and runtime passthrough.

Fixes #319

Bug fix

How Has This Been Tested?

  • uv run --extra openai --extra test pytest tests/inference_test.py -k gemini

Checklist:

  • I have read and acknowledged Google's Open Source Code of conduct.
  • I have read the Contributing page, and I either signed the Google Individual CLA or am covered by my company's Corporate CLA.
  • I have discussed my proposed solution with code owners in the linked issue(s) and we have agreed upon the general approach.
  • I have made any needed documentation changes, or noted in the linked issue(s) that documentation elsewhere needs updating.
  • I have added tests, or I have ensured existing tests cover the changes.
  • I have followed Google's Python Style Guide and ran pylint over the affected code.

@github-actions github-actions Bot added the size/XS Pull request with less than 50 lines changed label Mar 8, 2026
@google-cla

google-cla Bot commented Mar 8, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@autodots-AHassan
autodots-AHassan marked this pull request as draft March 8, 2026 22:31
@ROlwig

ROlwig commented Jun 4, 2026

Copy link
Copy Markdown

+1 — hit this exact bug in production today running LangExtract 1.5.0 with gemini-2.5-flash.

Symptom: Some lx.extract() calls returned 0 extractions with Skipping chunk: parse error: Failed to parse JSON content: Unterminated string starting at: line 1 column 698 (char 697). HTTP 200 from Gemini, but the JSON response body was truncated mid-string. Same prompt sometimes succeeded, sometimes truncated — i.e. thinking budget was the variable.

Confirmed: passing thinking_config via language_model_params was silently dropped by _API_CONFIG_KEYS filtering (line 287 + line 405), exactly as described in #319.

Workaround we shipped: import-time monkeypatch that adds 'thinking_config' to _API_CONFIG_KEYS, plus language_model_params={'max_output_tokens': 16384, 'thinking_config': ThinkingConfig(thinking_budget=0)}. The patch itself works (we can see thinking_config forwarded), but in our specific workload some complex prompts still hit truncation even with thinking_budget=0 — so we ultimately switched our LangExtract path to gemini-2.5-flash-lite (thinking off by default) while keeping gemini-2.5-flash for direct SDK calls.

The single-line allowlist addition in this PR is the right fix and would have saved us a day of debugging. Would love to see this land. Happy to share repro logs if useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XS Pull request with less than 50 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Add thinking_config to _API_CONFIG_KEYS for Gemini 3 model support

2 participants