Introduction

Running Ministral-3 on vLLM is surprisingly powerful. The model is fast, creative, and capable of producing high-quality responses even under heavy workloads.

But once you move from simple chat prompts to structured outputs, automation, or programmatic use, things quickly get complicated.

During the development of an AI-driven Jeopardy game with 12 simultaneous players and hundreds of model calls, we encountered several practical issues:

• Markdown appearing in plain-text responses
• JSON outputs becoming deeply nested
• Token budgets getting exceeded
• Small formatting differences breaking downstream logic

This guide summarizes the practical lessons learned while solving these issues, along with concrete patterns you can reuse in your own projects.

The Real-World Scenario

Our benchmark project simulates a full Jeopardy game where:

• The host AI generates categories and clues
• Multiple AI contestants answer simultaneously
• Another AI judge evaluates correctness
• Scores are tracked across rounds

A single game run can easily exceed 800 API calls.

This environment exposed edge cases that rarely appear in simple demos — making it a great test bed for understanding how Ministral behaves under real production workloads.

1. Serving Ministral-3 on vLLM

Ministral models do not use the standard HuggingFace tokenizer configuration.

This means the launch command must explicitly enable the Mistral tokenizer format.

vllm serve mistralai/Ministral-3-14B-Instruct-2512 \
  --tokenizer_mode mistral \
  --config_format mistral \
  --load_format mistral

If your application relies on function calling, add the tool flags:

--enable-auto-tool-choice
--tool-call-parser mistral

Important limitation

Unlike some other models, Ministral does not support chat_template_kwargs.

If you send a request like this:

{
  "chat_template_kwargs": {
    "enable_thinking": false
  }
}

vLLM returns:

HTTP 400: chat_template is not supported for Mistral tokenizers

That means features such as explicit “thinking mode” toggling (used with models like Qwen or DeepSeek) are simply not available.

Fortunately, this is rarely needed because Ministral already produces concise outputs by default.

2. Temperature: The Single Most Important Parameter

The official vLLM documentation consistently uses the following value with Ministral-3:

temperature = 0.15

At first glance this seems extremely low. However, it turns out to be critical for structured tasks.

What happens at higher temperatures

Using the OpenAI-style default:

temperature: 0.7

the model becomes overly creative with structure.

A simple request like:

{ "expertise": "2-3 topics they know best" }

may return something like:

{
  "expertise": [
    {
      "category": "Gourmet Pizza Alchemy",
      "detail": "Can transform random ingredients into Michelin-star pizza"
    },
    {
      "category": "Sumo Wrestling Physics",
      "detail": "Understands body mechanics and center-of-gravity combat"
    }
  ]
}

While technically valid JSON, it is not what the schema asked for.

The result:

• JSON becomes larger than expected
• max_tokens limits are exceeded
• Responses get truncated
• Parsing fails

Why 0.15 works better

At low temperature the model becomes structurally disciplined.

temperature: 0.15

Benefits:

• JSON follows schemas precisely
• token usage becomes predictable
• nested structures almost disappear
• responses stay within token limits

Even creative text generation remains strong — the model simply stops improvising with structure.

Recommendation: Use temperature: 0.15 as your default for Ministral-3.

3. Getting Clean JSON Responses

Producing machine-readable JSON from LLMs is harder than it sounds.

Ministral tends to interpret schema fields semantically instead of structurally, which leads to deeply nested output.

Return JSON with these fields.

{ "expertise": "2-3 topics they know best" }

{
  "expertise": [
    {
      "category": "Ancient Roman Engineering",
      "detail": "Knows aqueduct systems in surprising detail"
    },
    {
      "category": "Pizza Dough Chemistry",
      "detail": "Obsessed with yeast fermentation dynamics"
    }
  ]
}

Respond with ONLY valid JSON.
No markdown, no explanation, no text before or after the JSON.
Keep values as short plain strings — never use nested objects or arrays.

Every value MUST be a short plain string — NO arrays, NO nested objects.

function extractJSON(raw, shape) {
  var text = raw.replace(/^```(?:json)?\s*/i, '').replace(/\s*```$/i, '').trim();
  if (shape === 'array') {
    var m = text.match(/\[[\s\S]*\]/);
    if (m) text = m[0];
  } else {
    var m = text.match(/\{[\s\S]*\}/);
    if (m) text = m[0];
  }
  return text;
}

var stack = [];
var inStr = false, esc = false;

for (var i = 0; i < text.length; i++) {
  var ch = text[i];

  if (esc) { esc = false; continue; }
  if (ch === '\\') { esc = true; continue; }
  if (ch === '"') { inStr = !inStr; continue; }
  if (inStr) continue;

  if (ch === '{') stack.push('}');
  else if (ch === '[') stack.push(']');
  else if (ch === '}' || ch === ']') stack.pop();
}

text = text.replace(/,\s*$/, '');

while (stack.length > 0)
  text += stack.pop();

if (Array.isArray(value)) {
  flat = value.map(function(item) {
    if (typeof item === 'string') return item;
    if (typeof item === 'object') return Object.values(item).join(' — ');
    return String(item);
  }).join(', ');
}

2–3 retry attempts

verdict.toUpperCase().startsWith('CORRECT')

**CORRECT**

function stripMarkdown(text) {
  if (!text) return text;

  var s = text.replace(/\*\*([^*]+)\*\*/g, '$1');
  s = s.replace(/__([^_]+)__/g, '$1');
  s = s.replace(/\*([^*]+)\*/g, '$1');
  s = s.replace(/^#{1,6}\s+/gm, '');
  s = s.replace(/`([^`]+)`/g, '$1');
  s = s.replace(/^```[a-z]*\s*$/gm, '');

  return s.trim();
}

function buildModelProfile(modelName) {
  var lower = modelName.toLowerCase();
  var isMistral = lower.includes('mistral') || lower.includes('ministral');

  return {
    family: isMistral ? 'Mistral' : 'Generic',

    jsonSystemInstruction: isMistral
      ? 'Respond with ONLY valid JSON. No markdown. Keep values as short plain strings.'
      : 'You output only valid JSON. No markdown fences, no explanation.',

    jsonSchemaHint: isMistral
      ? ' Every value MUST be a short plain string — NO arrays, NO nested objects.'
      : '',

    jsonTemperature: isMistral ? 0.15 : 0.7,
    defaultTemperature: isMistral ? 0.15 : 0.7,

    plainTextInstruction: ' Do not use markdown formatting.'
  };
}