haraldwolff
/
whisper.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

whisper : make beam candidate sort more stable (#1943) 

All else being otherwise equal, this encourages the beam candidate
selection to re-use the same decoder, which slightly
reduces the cache size.

I wouldn't expect it to make much of a performance difference,
but it helps when debug printing the cache and beam.

Added as part of understanding #1941.
pull/1947/head
Josh Bleecher Snyder 2024-03-09 08:50:03 -08:00 committed by GitHub
parent ce945b50c3
commit 2852e1af55
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
whisper.cpp
View File

@ -5357,7 +5357,10 @@ int whisper_full_with_state(
beam_candidates.begin(),
beam_candidates.end(),
[](const beam_candidate & a, const beam_candidate & b) {
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
if (a.sequence.sum_logprobs_all != b.sequence.sum_logprobs_all) {
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
}
return a.decoder_idx < b.decoder_idx;
});
uint32_t cur_c = 0;