LLM API parameter glossary
80 API parameters appear across the catalog. This page defines each one, grouped by what it controls. Open any parameter for the full breakdown — its default, range, and conditions on every model that accepts it. Definitions come from the same community-maintained data as the JSON API.
Length
-
Max tokens
integer
max_tokens -
Maximum number of output tokens the model may generate.
175 models / Alibaba, Anthropic, Cohere, DeepSeek, Fireworks, MiniMax, Mistral, NVIDIA, OpenAI, Perplexity, Thinking Machines, Z.ai
-
Max completion tokens
integer
max_completion_tokens -
Maximum number of output tokens the model may generate.
100 models / Alibaba, Cerebras, Google, Groq, Meta, MiniMax, Moonshot AI, OpenAI, xAI, Xiaomi
-
Max tokens
integer
inferenceConfig.maxTokens -
Maximum number of output tokens the model may generate.
-
Stop sequences
string
inferenceConfig.stopSequences -
A list of strings where the model stops generating further tokens.
-
Stop sequence
string
stop -
Stops generation when this string is detected.
51 models / Cerebras, Groq, Mistral, Moonshot AI, NVIDIA, xAI, Xiaomi
-
Max output tokens
integer
generationConfig.maxOutputTokens -
Maximum number of tokens to include in a response candidate.
-
Candidate count
integer
generationConfig.candidateCount -
How many independent completions to generate for one request.
-
Stop sequences
string
generationConfig.stopSequences -
A list of strings where the model stops generating further tokens.
-
Stop sequences
string
stop_sequences -
Stops generation when one of these sequences is detected; up to five are allowed.
-
Number of completions
integer
n -
How many chat completion choices to generate for the request.
-
Max output tokens
integer
max_output_tokens -
Upper bound for output tokens generated in the Responses API response.
Sampling
-
Temperature
number
temperature -
Controls randomness. Lower values make outputs more focused; higher values make them more varied.
235 models / Alibaba, Anthropic, Cerebras, Cohere, DeepSeek, Fireworks, Google, Groq, Meta, MiniMax, Mistral, Moonshot AI, NVIDIA, OpenAI, Perplexity, Thinking Machines, xAI, Xiaomi, Z.ai
-
Top P
number
top_p -
Controls nucleus sampling by limiting generation to tokens within the selected cumulative probability.
226 models / Alibaba, Anthropic, Cerebras, DeepSeek, Fireworks, Google, Groq, Meta, MiniMax, Mistral, Moonshot AI, NVIDIA, OpenAI, Perplexity, Thinking Machines, xAI, Xiaomi, Z.ai
-
Frequency penalty
number
frequency_penalty -
Penalizes words based on how often they already appear in the generated text.
64 models / Cerebras, Cohere, Fireworks, Groq, Meta, Mistral, Moonshot AI, NVIDIA, Xiaomi
-
Presence penalty
number
presence_penalty -
Penalizes tokens that have already appeared to encourage a wider variety of content.
64 models / Cerebras, Cohere, Fireworks, Groq, Meta, Mistral, Moonshot AI, NVIDIA, Xiaomi
-
Top K
integer
top_k -
Limits token sampling to the top K most likely next tokens.
-
Temperature
number
inferenceConfig.temperature -
Controls randomness. Lower values make outputs more focused; higher values make them more varied. Bedrock rejects values above 1 for this model.
-
Top P
number
inferenceConfig.topP -
Controls nucleus sampling by limiting generation to tokens whose cumulative probability reaches this value.
-
Top K
integer
extra_body.top_k -
Limits generation to the selected number of highest-probability tokens.
-
Seed
integer
seed -
Seed used for best-effort deterministic sampling when reproducible outputs are desired.
-
Seed
integer
generationConfig.seed -
Optional seed used for decoding when reproducible sampling is desired.
-
Temperature
number
generationConfig.temperature -
Controls randomness. Lower values make outputs more focused; higher values make them more varied.
-
Top K
integer
generationConfig.topK -
Limits token sampling to the top K most likely next tokens.
-
Top P
number
generationConfig.topP -
Controls nucleus sampling by limiting generation to tokens within the selected cumulative probability.
-
Do sample
boolean
do_sample -
When false, the model uses greedy decoding and ignores temperature and top_p.
-
Random seed
integer
random_seed -
Seed used for deterministic sampling when reproducible outputs are desired.
-
Top K
integer
additionalModelRequestFields.top_k -
Limits token sampling to the top K most likely next tokens. Converse carries it through additionalModelRequestFields; it is not part of inferenceConfig.
-
Frequency penalty
number
generationConfig.frequencyPenalty -
Penalises tokens in proportion to how often they have already appeared.
-
Presence penalty
number
generationConfig.presencePenalty -
Penalises tokens that already appeared, pushing the model toward new topics.
-
Top K
integer
k -
Limits sampling to the K most likely tokens; 0 disables top-k sampling.
-
Top P
number
p -
Controls nucleus sampling by limiting generation to tokens within the selected cumulative probability.
-
Top K
integer
additionalModelRequestFields.inferenceConfig.topK -
Limits token sampling to the top K most likely next tokens. Nova nests it under an inferenceConfig object inside additionalModelRequestFields.
-
Repetition penalty
number
repetition_penalty -
Penalizes tokens that have already appeared to reduce repetition in the output.
Reasoning
-
Thinking mode
enum
thinking.type -
Controls the Anthropic thinking mode values supported by this model.
74 models / Anthropic, Cohere, DeepSeek, Moonshot AI, Xiaomi, Z.ai
-
Reasoning effort
enum
reasoning_effort -
Controls how much reasoning the model should perform before producing an answer.
52 models / Cerebras, DeepSeek, Groq, Meta, Moonshot AI, NVIDIA, OpenAI, Perplexity, Thinking Machines, xAI, Z.ai
-
Include thoughts
boolean
generationConfig.thinkingConfig.includeThoughts -
Controls whether Gemini returns available thought summaries in the response parts.
-
Budget tokens
integer
thinking.budget_tokens -
Maximum token budget Anthropic may use for extended thinking before producing the final answer.
-
Thinking level
enum
generationConfig.thinkingConfig.thinkingLevel -
Controls Gemini 3.1 Flash-Lite reasoning effort.
-
Thinking display
enum
thinking.display -
Controls whether Anthropic returns summarized or omitted thinking content.
-
Enable thinking
boolean
extra_body.enable_thinking -
Toggles the model's hybrid thinking mode, sent as a provider-specific extra body field on OpenAI-compatible clients.
-
Thinking budget
integer
extra_body.thinking_budget -
Maximum number of tokens the model may spend on reasoning before it starts the final answer; defaults to the model's maximum reasoning length.
-
Thinking budget
integer
generationConfig.thinkingConfig.thinkingBudget -
Token budget the model may spend on internal reasoning before answering. 0 disables thinking. Cannot be combined with thinkingLevel.
-
Enable thinking
boolean
extra_body.chat_template_kwargs.enable_thinking -
Controls Qwen3 thinking mode when using OpenAI-compatible clients that pass provider-specific extra body fields.
-
Effort
enum
output_config.effort -
Controls Anthropic response thoroughness and token spend.
-
Reasoning effort
enum
reasoning.effort -
Controls how much reasoning the model should perform before producing an answer.
-
Reasoning summary
enum
reasoning.summary -
Controls the level of reasoning summary returned with the response.
-
Split reasoning
boolean
reasoning_split -
Returns the model's reasoning in a separate reasoning_details field instead of inline with the response.
-
Budget tokens
integer
additionalModelRequestFields.thinking.budget_tokens -
Maximum token budget the model may use for extended thinking before producing the final answer. Must be less than inferenceConfig.maxTokens.
-
Thinking mode
enum
additionalModelRequestFields.thinking.type -
Controls Anthropic extended thinking on Bedrock. Bedrock rejects the adaptive mode the first-party Anthropic API accepts, so only disabled and enabled are listed here.
-
Reasoning format
enum
reasoning_format -
Controls how reasoning tokens are returned — hidden from the response, raw within the content, or parsed into a separate field.
-
Reasoning budget
integer
reasoning_budget -
Maximum number of tokens the model may use for internal reasoning before being forced to end the reasoning trace. Use -1 to disable budget enforcement.
-
Prompt mode
enum
prompt_mode -
Enables Mistral's reasoning system prompt; leave unset to disable the default reasoning behavior.
-
Keep prior reasoning
enum
thinking.keep -
Set to all to carry reasoning content from earlier assistant turns into the request; leave null to send only the final answers.
-
Clear thinking
boolean
clear_thinking -
When true, the model's thinking from previous turns is excluded from the conversation context; when false, it is preserved, which is useful for agentic workflows.
-
Separate reasoning
boolean
separate_reasoning -
Returns the model's reasoning in a dedicated reasoning_content field instead of interleaving it with the final message content.
-
Thinking token budget
integer
thinking.token_budget -
Maximum number of tokens the model may spend on reasoning before answering.
Tools
-
Tool choice
enum
tool_choice -
Controls whether the model may call tools, must call one, or skips tool calls.
21 models / Cohere, Meta, Moonshot AI, OpenAI
Output
-
Response format
enum
response_format.type -
Forces the response into plain text or a JSON object.
106 models / Cerebras, Cohere, Fireworks, Groq, Meta, Mistral, Moonshot AI, OpenAI, xAI, Xiaomi, Z.ai
-
Response MIME type
enum
generationConfig.responseMimeType -
MIME type for generated text candidates.
-
Verbosity
enum
text.verbosity -
Controls how concise or detailed the model's final text response should be.
-
Text format
enum
text.format.type -
Controls whether the Responses API returns free-form text, JSON mode output, or structured JSON schema output.
Observability
-
Log probabilities
boolean
logprobs -
Controls whether the response includes log probabilities for the generated tokens.
-
Top log probabilities
integer
top_logprobs -
Number of most likely tokens to return a log probability for at each position.
Metadata
-
Safe prompt
boolean
safe_prompt -
Controls whether Mistral injects its safety prompt before the conversation.
-
Safety mode
enum
safety_mode -
Controls Cohere's built-in safety instructions applied to the generation.
-
Return images
boolean
return_images -
Controls whether the response may include related images from the search.
-
Return related questions
boolean
return_related_questions -
Controls whether the response includes suggested follow-up questions.
-
Search after date
string
search_after_date_filter -
Restricts search results to content published after this date (MM/DD/YYYY).
-
Search before date
string
search_before_date_filter -
Restricts search results to content published before this date (MM/DD/YYYY).
-
Search domain filter
string
search_domain_filter -
Limits search to, or excludes, specific domains.
-
Search mode
enum
search_mode -
Selects the corpus the model searches when grounding its answer.
-
Search recency filter
enum
search_recency_filter -
Restricts web search results to a recent time window.
-
Search context size
enum
web_search_options.search_context_size -
Controls how much web search context is retrieved before generating the answer.
-
Disable search
boolean
disable_search -
Turns off web search so the model answers from its own knowledge only.
-
Prompt cache retention
enum
prompt_cache_retention -
Controls whether the prompt cache stays in memory or persists for up to 24 hours.
-
Chunk length
integer
chunk_length -
Context window size for processing. Longer texts are automatically split into chunks with overlap for complete coverage. Must be greater than overlap.
-
Expert type
enum
expert_type -
The type of expert to use. 'knowledge' answers with USD knowledge, 'code' responds with vanilla OpenUSD code, 'helperfunction' uses high-level helper functions, and 'auto' lets the LLM determine which expert to use.
-
Flat NER
boolean
flat_ner -
When true, prevents overlapping entity spans. When false, may return nested entities such as both a full name and its constituent first name.