r/LocalLLaMA • u/seamonn • 9h ago

New Model IBM Granite Speech 5.0 2B

https://huggingface.co/ibm-granite/granite-speech-5.0-2b
105 Upvotes

94% Upvoted

16 comments sorted by

24

u/seamonn 9h ago edited 9h ago

New Granite Speech Model from IBM. This one finally supports language input for LID and diarized pipelines.

Benchmarks well on the leaderboard (alongside mighty Qwen).

I am benchmarking it against Qwen 3 ASR 1.7B rn which has been my workhorse for transcription in my ASR pipeline.

Edit: In my quick smoke test, Qwen 3 ASR 1.7B still beats it so benchmarks are accurate. Still, this is a good alternative for those orgs who can't use Chinese models. It's a little behind Qwen is all. You can run but you can't escape the mighty Qwen.

2

u/hainesk 8h ago

I’ve been using Voxtral Mini 3B since its performed so well for me and I just saw Voxtral 3 came out yesterday. Have you tried Voxtral? Do you know how it compares to Qwen? I’ve generally found benchmarks to be misleading with ASR models.

6

u/SorosAhaverom 7h ago

fyi Voxtral 3 is closed-source, and from what I've seen there's no indication they will open source it

5

u/seamonn 8h ago

I have. It's very good in a transcription model only pipeline but nothing OS beats Qwen in a proper production pipeline.

The older Voxtral 2507 was generally better in my testing compared to the newer Voxtral Realtime.

The big Voxtral (24B) is too heavy to be run for ASR.

For reference, a proper ASR pipeline will have several stages:

  • Diarization (Nemotron 3 Diarization or Pyannote Community)
  • Speech Enhancement (This is likely bespoke)
  • LID (Language Identification)
  • Transcription (Qwen, Whisper, Voxtral)
  • Forced Alignment (Qwen)
  • Punctuation

1

u/Temporary-Career-360 3h ago

What is your setup?

1

u/coder543 8h ago

Did you see this? https://huggingface.co/inclusionAI/Ming-3.0-asr-tiny

It supposedly beats Qwen3-ASR-1.7B by a healthy margin.

2

u/seamonn 7h ago

I have been meaning to test it out but unfortunately, en and zh only

2

u/silenceimpaired 5h ago

I must be reading something wrong... it looks like their benchmarks say the last model was better.

1

u/Invader-Faye 2m ago

Its ibm, I wouldn't be surprised.

2

u/nuclearbananana 4h ago

Unfortunately it uses the same llm base model, which is just way too slow compared to parakeet and cohere.

Granite 4.1 also underperformed for me, especially for whispered speech. I will see how this one holds up.

2

u/nuclearbananana 2h ago

Update: 25x slower than parakeet and still worse. Not worth it by any measure

1

u/exaknight21 3h ago

I haven’t come around to trying parakeet yet, I tried Qwen ASR + my llama.cpp + kokoro for tts. I am struggling with response/gen on device (iPhone 16 base) i have access to 2 T4s. Do you recommend to try some way for fast ASR+TTS

2

u/nuclearbananana 3h ago

I'm not sure how you run on an iphone, I'm testing mainly on a laptop cpu and I've gotten the best results from https://github.com/handy-computer/transcribe.cpp

1

u/SmileyBMM 1h ago

Woah, how have I never heard of this before? I've been looking for something with newer models than Speech Note that still supports Vulkan. Thanks for mentioning it here.

2

u/nuclearbananana 50m ago

haha I also couldn't find it for a while. There's a couple other ggml, llama.cpp type things, but they don't have an accelerated cpu path like this one does so they were much slower for me than using onnx