Voxtral voice assistant

Use the official @open-wa/integration-voxtral plugin to answer WhatsApp voice notes and audio calls with a conversational assistant. Add it to your Easy API configuration, supply a Mistral API key, and describe how the assistant should respond.

The plugin handles transcription, conversation context, answer generation, and speech. Voice-note replies arrive as WhatsApp voice notes. Calls use a live audio connection, so callers can speak naturally and interrupt the assistant.

1. Install the pluginCopy Anchor Link

Use Node.js 22.21.1 or newer. Create a project and install Easy API and the plugin together:

mkdir whatsapp-voice-assistant
cd whatsapp-voice-assistant
npm init -y
npm pkg set type=module
npm install @open-wa/wa-automate @open-wa/integration-voxtral

Keep your Mistral API key in your environment:

export MISTRAL_API_KEY='YOUR_MISTRAL_API_KEY'

Mistral bills transcription, answer generation, and speech separately. The plugin uses your own Mistral account; see Mistral's current pricing.

2. Create config.jsCopy Anchor Link

Save this as config.js in the project directory:

export default {
  sessionId: 'voice-assistant',
  host: '127.0.0.1',
  port: 8080,
  dashboard: true,
  plugins: ['@open-wa/integration-voxtral'],
  pluginConfig: {
    voxtral: {
      mode: 'voice-notes',
      agent: {
        instructions: [
          'You are a helpful voice assistant.',
          'Reply in the language the person uses.',
          'Keep answers brief and easy to follow when spoken.',
          'Ask when you need more information.',
        ],
      },
    },
  },
};

If you already have an Easy API config, add the package to its plugins array and add pluginConfig.voxtral to its existing plugin settings.

The plugin reads MISTRAL_API_KEY automatically. Instructions can be a string or an array of strings; the plugin joins an array in order with line breaks. You can omit pluginConfig.voxtral entirely to use the default voice-note assistant. Add settings only when you want to change its behavior.

3. Start Easy APICopy Anchor Link

npx @open-wa/wa-automate --config ./config.js

Link WhatsApp using the QR code shown by Easy API, then open the dashboard and select Plugins → Voxtral. The plugin runs inside Easy API and uses its connected WhatsApp session. It keeps working when you close the dashboard.

Send a voice note to the linked account from another WhatsApp account. The assistant answers with a voice note. Send a follow-up such as “Can you explain the second point?” and it uses the earlier conversation to answer.

Choose voice notes, calls, or bothCopy Anchor Link

Set mode in pluginConfig.voxtral, or change Conversation mode on the Voxtral dashboard page:

The assistant handles direct conversations. It ignores groups, broadcasts, status updates, and its own messages. In voice-notes or both mode, you can also enable voiceNotes.acceptText to answer ordinary direct text messages.

Voice-note conversationsCopy Anchor Link

Each completed recording becomes a conversation turn. The plugin decrypts it, recognizes the speech, generates an answer, and sends prepared Ogg Opus audio using WhatsApp's voice-note presentation. You don't need to install FFmpeg or write a media handler.

The assistant remembers recent turns for the same person in the same WhatsApp session. Context expires after 30 minutes without activity by default and is cleared when Easy API restarts. Choose Clear AI context in the person's Chat view to clear a person's current context immediately.

To include a readable copy of the assistant's reply and let people type their follow-ups, add these settings inside voxtral:

voiceNotes: {
  includeText: true,
  acceptText: true,
},

includeText sends the assistant's answer as text alongside its voice note. It doesn't send the incoming transcript back as the answer.

Live audio callsCopy Anchor Link

Switch to calls or both and supply your calling-enabled OpenWA license in the top-level Easy API config:

export OPENWA_LICENSE_KEY='YOUR_CALLING_LICENSE'

export default {
  sessionId: 'voice-assistant',
  host: '127.0.0.1',
  port: 8080,
  dashboard: true,
  licenseKey: process.env.OPENWA_LICENSE_KEY,
  plugins: ['@open-wa/integration-voxtral'],
  pluginConfig: {
    voxtral: {
      mode: 'both',
      agent: {
        instructions: 'You are a helpful assistant. Give short, clear answers.',
      },
      calls: {
        answer: 'auto',
        greeting: 'Hello, you are speaking with our AI assistant. How can I help?',
      },
    },
  },
};

Start Easy API with --config ./config.js as before. Calling uses OpenWA's licensed audio calling support. If calling isn't available, the dashboard shows the reason; the Easy API and any enabled voice-note handling continue running.

With answer: 'auto', the assistant answers incoming calls and plays the greeting after the call connects. With answer: 'manual', incoming calls ring in the dashboard and you choose Answer with Voxtral, Answer myself, or Reject. To place a call, choose a contact in Calls, select Voxtral assistant as the audio source, and choose Call. In Chat, open the person and choose Call with Voxtral.

The caller can interrupt a spoken answer. The assistant stops its current speech, listens, and answers the new turn. Take over switches the live call to your microphone and speakers; Return to assistant gives the conversation back. These controls are also available in the dashboard's floating call control. OpenWA supports one active call per WhatsApp session.

During human takeover, the assistant can keep transcribing the caller. Your microphone is not transcribed. Add an optional handoff summary before returning the call to Voxtral if it needs facts from your side of the conversation.

In both mode, a person's voice notes and calls share recent conversation context. Voice notes received from that person during their call wait until the call ends, so the assistant doesn't answer them in two places at once.

Manage the assistant in the dashboardCopy Anchor Link

Open a person in Chat to see transcripts under their voice notes, assistant replies and delivery states, and call turns in the ordinary message timeline. The conversation toolbar shows the current assistant state and controls. Open Plugins → Voxtral for session settings, mode, voice preview, and usage; its conversation list links into the same Chat view.

The voice-note activity view shows Queued, Transcribing, Thinking, Generating voice, Sending, and the observed message acknowledgement. Calls show their connection state separately from Listening, Thinking, and Speaking, so you can tell whether the call connected and whether the assistant is producing audio.

Usage counters show the reporting period and provider values where available. Live transcription duration and interrupted requests may be missing; these counters are not a Mistral invoice.

Dashboard settings are saved for this session and override matching values in config.js after a restart. Reset to config.js removes those saved overrides. Use Copy config to move the effective settings into your file; the exported example references environment variables instead of including API keys. Voice and instruction changes apply to the next conversation turn. Mode changes affect new work; they don't silently hang up an active call.

Configuration referenceCopy Anchor Link

These fields belong in pluginConfig.voxtral:

Partial settings inherit defaults for omitted fields, including fields inside agent, voiceNotes, and calls. You don't need to copy a complete settings object to change one option. With language: 'auto', the plugin configures language detection for each provider request.

For better recognition of difficult recordings, set models.transcription to 'voxtral-small-transcribe-3-0'. Choose a voice suitable for the response language from Mistral's voice library. Model access and supported languages follow your Mistral account and the selected models. See the speech-to-text and text-to-speech references.

Conversation data and recoveryCopy Anchor Link

Recent transcripts and answers are held in memory for the conversation's idle timeout. The plugin doesn't save raw voice notes or call recordings by default. Settings, pause choices, usage counters, and message-processing identifiers are saved locally for the WhatsApp session. Audio and conversation text needed for responses are sent to Mistral.

If a voice note contains no recognizable speech, the assistant asks the person to send it again or type their message. If Mistral rejects a key or reports a limit, the dashboard shows the affected service and a recovery action. A failed reply isn't labelled as delivered, and the plugin doesn't automatically repeat a WhatsApp send whose result is uncertain.

If a live call loses its AI audio connection, the call control offers Take over, Reconnect assistant, and End call. Reconnecting starts a fresh AI audio connection on the existing call; it doesn't redial the person.

Related guidesCopy Anchor Link

• Easy API quick start
• Calls
• Media handling
• Plugin configuration