# inclusionAI/Ming-3.0-asr-tiny — vLLM serving recipe

> Chinese, English, and dialect speech recognition with topic, hotword, and dialogue context.

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `no_repeat_ngram`

- Interactive page: https://vllm-recipes-3ejlkdj3s-inferact-inc.vercel.app/inclusionAI/Ming-3.0-asr-tiny
- This configuration as JSON: https://vllm-recipes-3ejlkdj3s-inferact-inc.vercel.app/inclusionAI/Ming-3.0-asr-tiny.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `no_repeat_ngram` (default: `no_repeat_ngram`)

## Install

Requires vLLM >= nightly (not released yet — nightly wheel).

Install from main for now.

```bash
git clone --branch main https://github.com/vllm-project/vllm.git
cd vllm
uv venv --python 3.12
source .venv/bin/activate
VLLM_USE_PRECOMPILED=1 uv pip install --editable ".[audio]" --torch-backend=auto
vllm download-kernels
```

Docker image: `vllm/vllm-openai:nightly`

Extra dependencies:

- Ming uses torchaudio's sinc resampler; install a build compatible with PyTorch.

```bash
uv pip install torchaudio --torch-backend=auto
```

## Serve

```bash
vllm serve inclusionAI/Ming-3.0-asr-tiny \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --logits-processors vllm.model_executor.models.ming_v3_asr_logits_processor:NoRepeatNGramLogitsProcessor
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:nightly inclusionAI/Ming-3.0-asr-tiny \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --logits-processors vllm.model_executor.models.ming_v3_asr_logits_processor:NoRepeatNGramLogitsProcessor
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Ming-3.0-asr-tiny](https://huggingface.co/inclusionAI/Ming-3.0-asr-tiny) supports Chinese, English, and dialect transcription with optional topic, hotword, and dialogue context. 
This guide covers single-GPU serving, transcription, streaming, Context ASR, and offline inference with vLLM.

### Public example audio

Download the three clips from the official inference repository:

```bash
MING_AUDIO_BASE=https://raw.githubusercontent.com/inclusionAI/Ming-3.0-asr/6141e8c552815db724d8df0f72c9d865c547a1ea/examples/cases/audio
curl -fL "$MING_AUDIO_BASE/english_common_voice.wav" -o english.wav
curl -fL "$MING_AUDIO_BASE/chinese_common_voice.wav" -o chinese.wav
curl -fL "$MING_AUDIO_BASE/context_same_sentence_pcm16.wav" -o context.wav
```

| Clip | Observed transcription |
| --- | --- |
| English | `same way you did` |
| Mandarin | `漓江和湘江有秦代开凿的灵渠沟通` |
| Context clip with the hints below | Contains the film title `同囚` |

The Context clip illustrates correction of `铜球` to `同囚` with a relevant topic and hotwords. These are sample-specific observations, not a general Context ASR accuracy claim. Audio provenance and licenses are documented in the [official examples](https://github.com/inclusionAI/Ming-3.0-asr/blob/6141e8c552815db724d8df0f72c9d865c547a1ea/examples/cases/README.md).

For accuracy comparisons, keep batch settings and attention backends consistent; greedy transcription outputs can differ across these configurations.

### Transcription API

Use the audio requests below to verify speech recognition. The catalog's generic text-only cURL and benchmark shortcuts do not exercise ASR.

```bash
curl http://localhost:8000/v1/audio/transcriptions \
  -F model=inclusionAI/Ming-3.0-asr-tiny \
  -F file=@chinese.wav \
  -F temperature=0 \
  -F max_completion_tokens=256 \
  -F 'vllm_xargs={"no_repeat_ngram_size":10}'
```

Omit `language` for automatic recognition, or use `language=zh` / `language=en` to seed the known language. These are the supported language-hint codes; dialect transcription does not introduce additional hint codes. The response contains the transcript without the generated language label. `response_format=json` and `response_format=text` are supported.

#### OpenAI Python client

```python
import json
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
with open("english.wav", "rb") as audio:
    result = client.audio.transcriptions.create(
        model="inclusionAI/Ming-3.0-asr-tiny",
        file=audio,
        temperature=0,
        extra_body={
            "max_completion_tokens": 256,
            "vllm_xargs": json.dumps({"no_repeat_ngram_size": 10}),
        },
    )
print(result.text)
```

`vllm_xargs` is JSON-encoded for the multipart request, as in the cURL examples.

#### Context ASR

```bash
curl http://localhost:8000/v1/audio/transcriptions \
  -F model=inclusionAI/Ming-3.0-asr-tiny \
  -F file=@context.wav \
  -F language=zh \
  -F temperature=0 \
  -F max_completion_tokens=256 \
  -F 'prompt=This is an audio about 同囚.' \
  -F 'hotwords=同囚,黄国权,游学修,关楚耀,赵永洪,赵咏瑶,狱警,长镜头,吞玻璃戏,糖化玻璃,真实案例数据.' \
  -F 'vllm_xargs={"no_repeat_ngram_size":10,"context_chunks":true}'
```

`prompt` accepts topic or preceding dialogue appended to the official ASR instruction, or the full official instruction followed by context. Context and hotwords must be single-line text without audio placeholders. To reproduce the vendor's full prompt exactly, put its hotword text in `prompt` and omit the separate `hotwords` field.

#### Streaming

```bash
curl -N http://localhost:8000/v1/audio/transcriptions \
  -F model=inclusionAI/Ming-3.0-asr-tiny \
  -F file=@english.wav \
  -F language=en \
  -F temperature=0 \
  -F stream=true \
  -F max_completion_tokens=256 \
  -F 'vllm_xargs={"no_repeat_ngram_size":10}'
```

Streaming sends transcript deltas after the complete audio upload. It is not live microphone input. The English clip produces `same way you did` without leaking the language prefix into the transcript.

### Chunking and repetition control

Audio is downmixed and resampled to 16 kHz using torchaudio. The processor extracts the entire clip's log-mel features before splitting encoder chunks. Chunk settings are request-local and are included in multimodal cache keys.

| Option in `vllm_xargs` | Behavior |
| --- | --- |
| No chunk option | Use 3,000-mel-frame encoder chunks, approximately 30 seconds each. |
| `"context_chunks": true` | Use 3,000 frames up to 30 seconds and 6,000 frames for 30–60 seconds; reject longer clips. |
| `"audio_chunk_frames": 6000` | Explicit chunk size; any integer from 1 through 6,000 is accepted. |
| `"no_repeat_ngram_size": 10` | Block repeated 10-grams over prompt plus generated tokens. |

`context_chunks` and an explicit `audio_chunk_frames` are mutually exclusive. Ordinary chunking can handle clips beyond 60 seconds within the shared 8,192-token input/output budget and audio-upload size limit. Internal encoder chunking does not create separate transcription requests.

The no-repeat feature loads a plugin; each request must still set a nonzero `no_repeat_ngram_size` to enable blocking. Omitted values and `0` leave ordinary sampling in effect. Blocking uses the full prompt/output history, and requires Model Runner V2. Speculative decoding and MTP are unsupported with this plugin.

### Offline inference

From the same vLLM source checkout, use the shared [audio language model example](https://github.com/vllm-project/vllm/blob/main/examples/generate/multimodal/audio_language_offline.py) with its built-in audio assets:

```bash
python examples/generate/multimodal/audio_language_offline.py --model-type ming_v3_asr
```

This uses the shared example's sampling defaults (`temperature=0.2`, `max_tokens=64`) and prints raw model output, including the language label. For the downloaded Context clip, use the native prompt builder with greedy decoding and 10-gram blocking:

```python
from vllm import LLM, SamplingParams
from vllm.config.speech_to_text import SpeechToTextParams
from vllm.model_executor.models.ming_v3_asr import (
    MingV3ASRForConditionalGeneration as MingASR,
)
from vllm.multimodal.audio import AudioResampler
from vllm.multimodal.media.audio import load_audio

def main():
    llm = LLM(
        model="inclusionAI/Ming-3.0-asr-tiny",
        trust_remote_code=True,
        dtype="bfloat16",
        max_model_len=8192,
        max_num_seqs=4,
        enforce_eager=True,
        logits_processors=[
            "vllm.model_executor.models.ming_v3_asr_logits_processor:"
            "NoRepeatNGramLogitsProcessor",
        ],
    )
    audio, rate = load_audio("context.wav", sr=None, mono=True)
    audio = AudioResampler(16000).resample(audio, orig_sr=rate)
    params = SpeechToTextParams(
        audio=audio,
        stt_config=MingASR.get_speech_to_text_config(llm.model_config, "transcribe"),
        model_config=llm.model_config,
        language="zh",
        request_prompt="This is an audio about 同囚.",
        hotwords=(
            "同囚,黄国权,游学修,关楚耀,赵永洪,赵咏瑶,狱警,长镜头,"
            "吞玻璃戏,糖化玻璃,真实案例数据."
        ),
        extra_args={"context_chunks": True},
    )
    outputs = llm.generate(
        MingASR.get_generation_prompt(params),
        SamplingParams(
            temperature=0,
            max_tokens=256,
            extra_args={"no_repeat_ngram_size": 10},
        ),
    )
    print(MingASR.post_process_output(outputs[0].outputs[0].text))


if __name__ == "__main__":
    main()
```

### Limitations

The [official model card](https://huggingface.co/inclusionAI/Ming-3.0-asr-tiny#usage-notes) describes transcription text output without word-level timestamps or speaker diarization.

The current vLLM integration does not support translation, segment timestamps, or live audio input. Translation and timestamp requests are rejected. The `context_chunks=true` policy accepts clips up to 60 seconds; ordinary chunking, including requests with context prompts, can accept longer clips subject to the model context and upload limits. Longer recordings can produce incomplete transcripts; accepting an upload does not guarantee complete transcription.

### References

- [model card](https://huggingface.co/inclusionAI/Ming-3.0-asr-tiny)
- [Official inference repository](https://github.com/inclusionAI/Ming-3.0-asr)
- [vLLM speech-to-text API](https://docs.vllm.ai/en/latest/serving/online_serving/speech_to_text.html)