{
  "hf_id": "inclusionAI/Ming-3.0-asr-tiny",
  "meta": {
    "title": "Ming3-ASR-tiny",
    "slug": "ming-3-0-asr-tiny",
    "provider": "inclusionAI",
    "description": "Chinese, English, and dialect speech recognition with topic, hotword, and dialogue context.",
    "date_added": "2026-10-26",
    "date_updated": "2026-10-27",
    "tasks": [
      "multimodal"
    ],
    "performance_headline": "Context-aware Chinese, English, and dialect ASR; single-GPU serving"
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai:nightly",
    "command": "vllm serve inclusionAI/Ming-3.0-asr-tiny \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1 \\\n  --logits-processors vllm.model_executor.models.ming_v3_asr_logits_processor:NoRepeatNGramLogitsProcessor",
    "argv": [
      "vllm",
      "serve",
      "inclusionAI/Ming-3.0-asr-tiny",
      "--trust-remote-code",
      "--tensor-parallel-size",
      "1",
      "--logits-processors",
      "vllm.model_executor.models.ming_v3_asr_logits_processor:NoRepeatNGramLogitsProcessor"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai:nightly inclusionAI/Ming-3.0-asr-tiny \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1 \\\n  --logits-processors vllm.model_executor.models.ming_v3_asr_logits_processor:NoRepeatNGramLogitsProcessor",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai:nightly",
      "inclusionAI/Ming-3.0-asr-tiny",
      "--trust-remote-code",
      "--tensor-parallel-size",
      "1",
      "--logits-processors",
      "vllm.model_executor.models.ming_v3_asr_logits_processor:NoRepeatNGramLogitsProcessor"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "kv_store_centralized_mooncake": "/inclusionAI/Ming-3.0-asr-tiny/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/inclusionAI/Ming-3.0-asr-tiny/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/inclusionAI/Ming-3.0-asr-tiny/hw/h200.json",
      "h100": "/inclusionAI/Ming-3.0-asr-tiny/hw/h100.json",
      "b200": "/inclusionAI/Ming-3.0-asr-tiny/hw/b200.json",
      "gb200": "/inclusionAI/Ming-3.0-asr-tiny/hw/gb200.json",
      "b300": "/inclusionAI/Ming-3.0-asr-tiny/hw/b300.json",
      "gb300": "/inclusionAI/Ming-3.0-asr-tiny/hw/gb300.json",
      "mi300x": "/inclusionAI/Ming-3.0-asr-tiny/hw/mi300x.json",
      "mi325x": "/inclusionAI/Ming-3.0-asr-tiny/hw/mi325x.json",
      "mi350x": "/inclusionAI/Ming-3.0-asr-tiny/hw/mi350x.json",
      "mi355x": "/inclusionAI/Ming-3.0-asr-tiny/hw/mi355x.json"
    }
  },
  "model": {
    "model_id": "inclusionAI/Ming-3.0-asr-tiny",
    "min_vllm_version": "nightly",
    "nightly_required": true,
    "architecture": "moe",
    "parameter_count": "8.55B",
    "active_parameters": "1.3B",
    "context_length": 8192,
    "base_args": [
      "--trust-remote-code"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "git clone --branch main https://github.com/vllm-project/vllm.git\ncd vllm\nuv venv --python 3.12\nsource .venv/bin/activate\nVLLM_USE_PRECOMPILED=1 uv pip install --editable \".[audio]\" --torch-backend=auto\nvllm download-kernels\n",
        "note": "Install from main for now."
      },
      "extras": [
        {
          "note": "Ming uses torchaudio's sinc resampler; install a build compatible with PyTorch.",
          "command": "uv pip install torchaudio --torch-backend=auto"
        }
      ]
    }
  },
  "features": {
    "no_repeat_ngram": {
      "description": "Load the Model Runner V2 plugin for request-level full-history n-gram blocking.",
      "args": [
        "--logits-processors",
        "vllm.model_executor.models.ming_v3_asr_logits_processor:NoRepeatNGramLogitsProcessor"
      ]
    }
  },
  "opt_in_features": [],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 22,
      "description": "BF16 checkpoint; the capacity estimate includes the audio tower and is not a measured minimum."
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\n[Ming-3.0-asr-tiny](https://huggingface.co/inclusionAI/Ming-3.0-asr-tiny) supports Chinese, English, and dialect transcription with optional topic, hotword, and dialogue context. \nThis guide covers single-GPU serving, transcription, streaming, Context ASR, and offline inference with vLLM.\n\n## Public example audio\n\nDownload the three clips from the official inference repository:\n\n```bash\nMING_AUDIO_BASE=https://raw.githubusercontent.com/inclusionAI/Ming-3.0-asr/6141e8c552815db724d8df0f72c9d865c547a1ea/examples/cases/audio\ncurl -fL \"$MING_AUDIO_BASE/english_common_voice.wav\" -o english.wav\ncurl -fL \"$MING_AUDIO_BASE/chinese_common_voice.wav\" -o chinese.wav\ncurl -fL \"$MING_AUDIO_BASE/context_same_sentence_pcm16.wav\" -o context.wav\n```\n\n| Clip | Observed transcription |\n| --- | --- |\n| English | `same way you did` |\n| Mandarin | `漓江和湘江有秦代开凿的灵渠沟通` |\n| Context clip with the hints below | Contains the film title `同囚` |\n\nThe Context clip illustrates correction of `铜球` to `同囚` with a relevant topic and hotwords. These are sample-specific observations, not a general Context ASR accuracy claim. Audio provenance and licenses are documented in the [official examples](https://github.com/inclusionAI/Ming-3.0-asr/blob/6141e8c552815db724d8df0f72c9d865c547a1ea/examples/cases/README.md).\n\nFor accuracy comparisons, keep batch settings and attention backends consistent; greedy transcription outputs can differ across these configurations.\n\n## Transcription API\n\nUse the audio requests below to verify speech recognition. The catalog's generic text-only cURL and benchmark shortcuts do not exercise ASR.\n\n```bash\ncurl http://localhost:8000/v1/audio/transcriptions \\\n  -F model=inclusionAI/Ming-3.0-asr-tiny \\\n  -F file=@chinese.wav \\\n  -F temperature=0 \\\n  -F max_completion_tokens=256 \\\n  -F 'vllm_xargs={\"no_repeat_ngram_size\":10}'\n```\n\nOmit `language` for automatic recognition, or use `language=zh` / `language=en` to seed the known language. These are the supported language-hint codes; dialect transcription does not introduce additional hint codes. The response contains the transcript without the generated language label. `response_format=json` and `response_format=text` are supported.\n\n### OpenAI Python client\n\n```python\nimport json\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\nwith open(\"english.wav\", \"rb\") as audio:\n    result = client.audio.transcriptions.create(\n        model=\"inclusionAI/Ming-3.0-asr-tiny\",\n        file=audio,\n        temperature=0,\n        extra_body={\n            \"max_completion_tokens\": 256,\n            \"vllm_xargs\": json.dumps({\"no_repeat_ngram_size\": 10}),\n        },\n    )\nprint(result.text)\n```\n\n`vllm_xargs` is JSON-encoded for the multipart request, as in the cURL examples.\n\n### Context ASR\n\n```bash\ncurl http://localhost:8000/v1/audio/transcriptions \\\n  -F model=inclusionAI/Ming-3.0-asr-tiny \\\n  -F file=@context.wav \\\n  -F language=zh \\\n  -F temperature=0 \\\n  -F max_completion_tokens=256 \\\n  -F 'prompt=This is an audio about 同囚.' \\\n  -F 'hotwords=同囚,黄国权,游学修,关楚耀,赵永洪,赵咏瑶,狱警,长镜头,吞玻璃戏,糖化玻璃,真实案例数据.' \\\n  -F 'vllm_xargs={\"no_repeat_ngram_size\":10,\"context_chunks\":true}'\n```\n\n`prompt` accepts topic or preceding dialogue appended to the official ASR instruction, or the full official instruction followed by context. Context and hotwords must be single-line text without audio placeholders. To reproduce the vendor's full prompt exactly, put its hotword text in `prompt` and omit the separate `hotwords` field.\n\n### Streaming\n\n```bash\ncurl -N http://localhost:8000/v1/audio/transcriptions \\\n  -F model=inclusionAI/Ming-3.0-asr-tiny \\\n  -F file=@english.wav \\\n  -F language=en \\\n  -F temperature=0 \\\n  -F stream=true \\\n  -F max_completion_tokens=256 \\\n  -F 'vllm_xargs={\"no_repeat_ngram_size\":10}'\n```\n\nStreaming sends transcript deltas after the complete audio upload. It is not live microphone input. The English clip produces `same way you did` without leaking the language prefix into the transcript.\n\n## Chunking and repetition control\n\nAudio is downmixed and resampled to 16 kHz using torchaudio. The processor extracts the entire clip's log-mel features before splitting encoder chunks. Chunk settings are request-local and are included in multimodal cache keys.\n\n| Option in `vllm_xargs` | Behavior |\n| --- | --- |\n| No chunk option | Use 3,000-mel-frame encoder chunks, approximately 30 seconds each. |\n| `\"context_chunks\": true` | Use 3,000 frames up to 30 seconds and 6,000 frames for 30–60 seconds; reject longer clips. |\n| `\"audio_chunk_frames\": 6000` | Explicit chunk size; any integer from 1 through 6,000 is accepted. |\n| `\"no_repeat_ngram_size\": 10` | Block repeated 10-grams over prompt plus generated tokens. |\n\n`context_chunks` and an explicit `audio_chunk_frames` are mutually exclusive. Ordinary chunking can handle clips beyond 60 seconds within the shared 8,192-token input/output budget and audio-upload size limit. Internal encoder chunking does not create separate transcription requests.\n\nThe no-repeat feature loads a plugin; each request must still set a nonzero `no_repeat_ngram_size` to enable blocking. Omitted values and `0` leave ordinary sampling in effect. Blocking uses the full prompt/output history, and requires Model Runner V2. Speculative decoding and MTP are unsupported with this plugin.\n\n## Offline inference\n\nFrom the same vLLM source checkout, use the shared [audio language model example](https://github.com/vllm-project/vllm/blob/main/examples/generate/multimodal/audio_language_offline.py) with its built-in audio assets:\n\n```bash\npython examples/generate/multimodal/audio_language_offline.py --model-type ming_v3_asr\n```\n\nThis uses the shared example's sampling defaults (`temperature=0.2`, `max_tokens=64`) and prints raw model output, including the language label. For the downloaded Context clip, use the native prompt builder with greedy decoding and 10-gram blocking:\n\n```python\nfrom vllm import LLM, SamplingParams\nfrom vllm.config.speech_to_text import SpeechToTextParams\nfrom vllm.model_executor.models.ming_v3_asr import (\n    MingV3ASRForConditionalGeneration as MingASR,\n)\nfrom vllm.multimodal.audio import AudioResampler\nfrom vllm.multimodal.media.audio import load_audio\n\ndef main():\n    llm = LLM(\n        model=\"inclusionAI/Ming-3.0-asr-tiny\",\n        trust_remote_code=True,\n        dtype=\"bfloat16\",\n        max_model_len=8192,\n        max_num_seqs=4,\n        enforce_eager=True,\n        logits_processors=[\n            \"vllm.model_executor.models.ming_v3_asr_logits_processor:\"\n            \"NoRepeatNGramLogitsProcessor\",\n        ],\n    )\n    audio, rate = load_audio(\"context.wav\", sr=None, mono=True)\n    audio = AudioResampler(16000).resample(audio, orig_sr=rate)\n    params = SpeechToTextParams(\n        audio=audio,\n        stt_config=MingASR.get_speech_to_text_config(llm.model_config, \"transcribe\"),\n        model_config=llm.model_config,\n        language=\"zh\",\n        request_prompt=\"This is an audio about 同囚.\",\n        hotwords=(\n            \"同囚,黄国权,游学修,关楚耀,赵永洪,赵咏瑶,狱警,长镜头,\"\n            \"吞玻璃戏,糖化玻璃,真实案例数据.\"\n        ),\n        extra_args={\"context_chunks\": True},\n    )\n    outputs = llm.generate(\n        MingASR.get_generation_prompt(params),\n        SamplingParams(\n            temperature=0,\n            max_tokens=256,\n            extra_args={\"no_repeat_ngram_size\": 10},\n        ),\n    )\n    print(MingASR.post_process_output(outputs[0].outputs[0].text))\n\n\nif __name__ == \"__main__\":\n    main()\n```\n\n## Limitations\n\nThe [official model card](https://huggingface.co/inclusionAI/Ming-3.0-asr-tiny#usage-notes) describes transcription text output without word-level timestamps or speaker diarization.\n\nThe current vLLM integration does not support translation, segment timestamps, or live audio input. Translation and timestamp requests are rejected. The `context_chunks=true` policy accepts clips up to 60 seconds; ordinary chunking, including requests with context prompts, can accept longer clips subject to the model context and upload limits. Longer recordings can produce incomplete transcripts; accepting an upload does not guarantee complete transcription.\n\n## References\n\n- [model card](https://huggingface.co/inclusionAI/Ming-3.0-asr-tiny)\n- [Official inference repository](https://github.com/inclusionAI/Ming-3.0-asr)\n- [vLLM speech-to-text API](https://docs.vllm.ai/en/latest/serving/online_serving/speech_to_text.html)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "a39b1759dba64342a9262a640c46f63abc0e2d6c",
    "path": "recipes/models/inclusionAI/Ming-3.0-asr-tiny.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/inclusionAI/Ming-3.0-asr-tiny.yaml",
    "overlays": []
  }
}