59fdcd19c8
whisper : add whisper_state + default state on the whisper_context ( #523 )
...
* Added whisper state + default state on the whisper_context
* Fixed some examples and bindings
* Fixed whisper_n_len (which was used in some binding) and added whisper_n_len_from_state
* Fixed comments
* whisper : reuse kv_cache_free() and fix compiler warnings
* whisper : clean-up the API comments
---------
Co-authored-by: Sandro Hanea <sandrohanea@microsoft.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-03-05 21:42:19 +02:00
478289a4b3
whisper : set no_context == true by default ( #537 )
2023-03-05 20:53:43 +02:00
373043cabe
whisper : zero-initialize some more context variables
...
Just in case
2023-02-21 19:00:42 +02:00
fb4d0d470f
whisper : fix uninitialized exp_n_audio_ctx
2023-02-21 18:58:08 +02:00
0d229163bb
whisper : add API for applying custom logits filters during decoding
2023-02-19 18:35:01 +02:00
a94897bcde
whisper : by default disable non-speech tokens suppression ( #473 )
...
This seems to be causing hallucinations in the end of the audio, e.g.:
"Thank you for listening"
"Amen"
..
2023-02-15 21:48:49 +02:00
0336161b7d
whisper : fix signedness compiler warning ( #506 )
2023-02-15 19:08:25 +02:00
cfc06bf8df
whisper : suppress non-speech-related token outputs ( #473 )
...
* add non-speech-token suppression
* add suppress non-speech_tokens param
2023-02-08 09:05:34 +02:00
2bfe0ebc0f
whisper : fixed Beam Search Strategy and exposed whisper_pcm_to_mel_phase_vocoder ( #474 )
...
Co-authored-by: Sandro Hanea <sandrohanea@microsoft.com>
2023-02-08 09:01:47 +02:00
4dd7119deb
whisper : only trim if split_on_word is true ( #476 )
2023-02-08 08:43:23 +02:00
a1c1583cc7
whisper : add whisper_full_lang_id() for getting the context lang ( #461 )
2023-02-05 14:46:26 +02:00
d012b5c7e4
whisper : add "split_on_word" flag when using using "max_len" option ( #455 )
...
* Update whisper.cpp
* fix: trim function
* feat: added flag to split on word
* fix: arguments for main
2023-02-05 14:44:23 +02:00
f3ee4a9673
whisper : reduce memory usage during inference ( #431 )
...
* ggml : add "scratch" buffer support
* ggml : support for scratch ring-buffer
* ggml : bug fix in ggml_repeat()
* ggml : error on scratch buffer overflow
* whisper : use scratch buffers during inference (base model only)
* whisper : update memory usage for all models
* whisper : fix encoder memory usage
* whisper : use whisper_context functions instead of macros
* whisper : fix FF + remove it from README
* ggml : reuse ggml_new_i32
* ggml : refactor the scratch buffer storage
* whisper : reorder scratch buffers in the decoder
* main : add option to disable temp fallback
* Update README.md
2023-02-04 09:45:52 +02:00
291980369c
whisper : suppress task tokens ( #442 )
2023-02-04 09:03:14 +02:00
b992f3709e
whisper : do not provide past prompt when n_max_text_ctx == 0
2023-01-25 20:01:00 +02:00
b5ddb16ec7
whisper : condition timestamps to be monotonically increasing ( #425 )
2023-01-23 20:48:26 +02:00
ae16c21e9c
whisper : PPC64 big-endian support ( #398 )
...
* ggml : set cache line size to 128 on POWER9
* whisper : add PPC64 big endian support
2023-01-23 20:48:10 +02:00
78f166174f
whisper : fix condition for providing past prompt (critical)
...
This bug has been present since v1.1.0.
Effectively, the past transcribed text wasn't being used for following
transcriptions, which likely significantly reduces the transcription
quality.
Likely related to #419
2023-01-22 10:47:01 +02:00
21c569ba4a
whisper : extend information in whisper_print_timings()
2023-01-19 18:50:33 +02:00
1a91c19af9
whisper : perform entropy check only when we have at least 32 tokens ( #412 )
2023-01-18 22:52:18 +02:00
a6cf6f4c4a
bench : minor fixes
2023-01-18 21:40:10 +02:00
1ccb8a46a5
bench : fix Windows linkage by moving ggml benches in whisper lib ..
2023-01-18 21:19:50 +02:00
8088a977af
whisper : fix possible uninitialized variables ( #291 )
2023-01-16 21:44:40 +02:00
00ea21668b
whisper : account speed_up flag for short audio ( close #405 )
2023-01-15 12:42:15 +02:00
8de452c18b
Improve decoding ( #291 )
...
* whisper : prepare infra for new decoding strategies
* whisper : apply logit filters and compute logprobs
* whisper : add whisper_get_logits()
* whisper : separate self and cross attention memory
Initial step needed for supporting parallel decoders
* whisper : move probs_id buffer to whisper_context
* whisper : refactor kv cache into separate struct
* whisper : move self-attention kv cache to whisper_decoder
* whisper : wip decoding parameters + strategies
* whisper : wip decoding parameters + strategies (part 2)
* whisper : wip decoding parameters + strategies (part 3)
* whisper : wip decoding parameters + strategies (part 4)
* whisper : fix prompt_past update to not include prompt_init
* whisper : temperature + best_of support
* whisper : support for compression_ration_threshold
We actually use entropy, but it is similar
* command : fix example to use logits instead of obsolete probs
* whisper : handle empty sequence ranking
* whisper : add WHISPER_DEBUG + diagnostic prints + new main args
* whisper : minor fixes
* whisper : add beam-search support
* whisper : bug fix when there no previous context
* whisper : add comments
* stream : disable temperature fallback
For real-time processing, we always want a single decoder running at T=0
* whisper.swiftui : update example - fix paths + add empty folders
2023-01-15 11:29:57 +02:00
4ef3398e8f
ggml : remove obsolete zeroing + comment fixes ( #390 )
2023-01-08 20:21:03 +02:00
08dc705a69
whisper : fix sample_to_timestamp calculation with 64 bit precision to avoid overflow ( #388 )
...
* Do calculation with 64 bit precision to avoid overflow
* Update whisper.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-01-08 15:08:45 +02:00
1512545149
whisper : add loader class to allow loading from buffer and others ( #353 )
...
* whisper : add loader to allow loading from other than file
* whisper : rename whisper_init to whisper_init_from_file
* whisper : add whisper_init_from_buffer
* android : Delete local.properties
* android : load models directly from assets
* whisper : adding <stddef.h> needed for size_t + code style
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-01-08 13:03:33 +02:00
65fdcbbbbb
whisper : revert accidental MB change
2023-01-07 16:18:21 +02:00
d61d55cd4b
ggml : speed-up soft max via Accelerate + unroll
2023-01-07 16:16:42 +02:00
a62170c656
ggml : add SSE3 and fp16 conversion lookup table ( #368 )
...
* Improves WASM performance:
On MacBook M1 Pro, I observe 25% faster using Firefox and 35% faster using Chrome
* Add support for SSE3 SIMD
* Add SSE3 to system information
* Add Imath support for fp16-fp32 conversions
* Add Imath to system information
* Wrap Imath calls to avoid static function warnings
* Drop Imath; Add lookup table for f16 -> f32 conversions
* Remove TODO comments
* Update SSE3 to new macro arguments
* Correct updated macro definitions
* Prefer static inline where possible
* ggml : static inlines + add public f16 <-> f32 conversions
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-01-06 18:45:59 +02:00
1944e7c33e
whisper : document POWER VSX support
2023-01-05 23:53:00 +02:00
ad2a4ffa03
whisper : do not use F16 tensors when in F32 mode ( #369 )
2023-01-05 22:56:25 +02:00
dd6d582977
whisper : use ranged-based for loops for readability
2023-01-05 21:20:44 +02:00
d51c5eb906
ggml : define MIN / MAX only if not defined (minor)
2023-01-05 21:16:52 +02:00
d97e6005e9
whisper : add whisper_n_audio_ctx and check for invalid audio_ctx
...
closes #344
2022-12-31 09:57:19 +02:00
68daf6e487
whisper : avoid some memory allocations
2022-12-30 13:43:48 +02:00
ac521a566e
ggml : simplify the SIMD code ( #324 )
...
* ggml : simplify the SIMD code
* ggml : generic reduce for all register sizes + comments
2022-12-24 10:22:28 +02:00
543bd5627e
whisper : use emplace_back in place of push_back ( #319 )
...
This avoids potential construction of temporaries.
2022-12-23 11:07:19 +02:00
62fee9a9cc
whisper : fix mem leak on failure to load model ( #318 )
2022-12-23 11:06:17 +02:00
fa463313ad
minor : small code cleanups ( #302 )
...
* Small code cleanups
- fix indentation
- remove extra semicolons
- remove extra break after returns in case statements
- remove unnecessary call to .data() on string
- use empty() instead of checking size()
- no need to check for nullptr before free
- remove unnecessary initialization of string to ""
* minor : switch case always break
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2022-12-22 17:06:19 +02:00
501a6b455c
minor : flag "ARM FMA" -> "ARM_FMA"
2022-12-22 16:47:54 +02:00
e1432dd91a
Check for both __ARM_NEON and __ARM_FEATURE_FMA so that the project can be compiled for armv7a.
...
Android armeabi-v7a's NEON support doesn't support FMA unless configured with `-mfpu=neon-fp-armv8`, which would need runtime checks.
* Also removed ABI filter from Android project.
2022-12-22 16:47:54 +02:00
42c6730732
whisper : use nullptr (C++11) instead of NULL macro ( #299 )
2022-12-22 16:35:18 +02:00
99da1e5cc8
cmake : enable and fix -Wall -Wextra -Wpedantic C++ warnings
2022-12-19 20:45:08 +02:00
8e3f129b4d
minor : resolves some of warnings when compiling with clang/clang++ ( #294 )
...
* Resolves some of warnings when compiling with clang/clang++
Mostly nit stuff that clang catches when compiling with -Wall -Wextra
-pedantic.
- Fix comparison between sign/unsigned integers.
- Passes a constant reference (const&) instead of copying each time.
* minor : normalize coding style
* minor : fix warning
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2022-12-19 20:19:01 +02:00
fba10a4c68
whisper : language auto-detect ( #59 )
2022-12-17 18:49:44 +02:00
6a69e3ae27
command : adding guided mode
2022-12-16 19:38:18 +02:00
bf69b669a0
whisper : add whisper_tokenize()
...
Tokenizes a string into a list of vocabulary tokens
2022-12-16 19:38:18 +02:00
6a7c82501e
whisper : improve decoding strategy ( #244 )
...
- Clear past prompt when there is very short audio left for processing.
My observation is that in these cases the decoding tends to repeat and
hallucinate stuff and I think this is induced by the existing prompt
- When we fail to sample timestamp token, retry by clearing the past
prompt. If it fails again, then we advance the window by 1 second
2022-12-16 18:34:35 +02:00
sandrohanea
Georgi Gerganov
Finn Voorhees
shikokuchuo
shibukazu
boolemancer
kamranjon
Matija Pevec
fitzsim
Syahmi Azhar
Abitofevrything
Andy Maloney
Kevin Brothaler
Matheus de Sousa