haraldwolff/whisper.cpp
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
535 commits 41 branches 23 tags 85 MiB
Commit graph

128 commits

Author SHA1 Message Date
Georgi Gerganov 514cd04452 whisper : fix bug in prompt processing (close #705) 
Was dereferencing a dangling pointer
 2023-04-14 19:17:07 +03:00
Georgi Gerganov 69b8503935
ggml : backport llama.cpp updates (close #709) 
- About x2 overall performance improvement on Apple Silicon
- Results should now be the same for different number of threads (not
  tested)
 2023-04-10 22:28:54 +03:00
pajowu 0a2d1210bc
whisper : add progress callback (#600) 2023-03-30 20:29:29 +03:00
Jhen-Jie Hong eefed45e37
whisper : add initial_prompt param (#645) 2023-03-29 23:23:23 +03:00
Georgi Gerganov 42c6855103
whisper : bump "large" scratch buffer even mode (close #671) 2023-03-28 10:50:49 +03:00
Georgi Gerganov 0be9cd3497
whisper : increase scratch buffers after recent change (#671) 
Should fix the error:

ggml_new_tensor_impl: not enough space in the scratch memory
 2023-03-28 10:36:16 +03:00
Georgi Gerganov 4a0deb8b1e
talk-llama : add new example + sync ggml from llama.cpp (#664) 
* talk-llama : talk with LLaMA AI

* talk.llama : disable EOS token

* talk-llama : add README instructions

* ggml : fix build in debug
 2023-03-27 21:00:32 +03:00
Georgi Gerganov 8e361d90d7
whisper : disable fallbacks until the performance is improved (#588) 2023-03-22 22:34:39 +02:00
sandrohanea d4fa0d92ad
fixed language auto-detection for state provided processing (#627) 
Co-authored-by: Sandro Hanea <sandrohanea@microsoft.com>
 2023-03-22 21:47:09 +02:00
Leo Moll 8fcd1a3b32
main : provide option for creating JSON output (#615) 
* examples : provide option for exporting also as JSON file (ggerganov/whisper.cpp#614)

* main : remove leftovers

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-22 21:37:36 +02:00
Georgi Gerganov 1beff6f66d
models : change HF hosting from dataset to model 2023-03-22 20:44:56 +02:00
Takeshi Inoue 09e9068007
whisper.android : support benchmark for Android example. (#542) 
* whisper.android: Support benchmark for Android example.

* whisper.android: update screenshot in README.

* update: Make text selectable for copy & paste.

* Update whisper.h to restore API name

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* whisper.android: Restore original API names.

---------

Co-authored-by: tinoue <tinoue@xevo.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-07 21:36:30 +02:00
sandrohanea 59fdcd19c8
whisper : add whisper_state + default state on the whisper_context (#523) 
* Added whisper state + default state on the whisper_context

* Fixed some examples and bindings

* Fixed whisper_n_len (which was used in some binding) and added whisper_n_len_from_state

* Fixed comments

* whisper : reuse kv_cache_free() and fix compiler warnings

* whisper : clean-up the API comments

---------

Co-authored-by: Sandro Hanea <sandrohanea@microsoft.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-05 21:42:19 +02:00
Georgi Gerganov 478289a4b3
whisper : set no_context == true by default (#537) 2023-03-05 20:53:43 +02:00
Georgi Gerganov 373043cabe
whisper : zero-initialize some more context variables 
Just in case
 2023-02-21 19:00:42 +02:00
Finn Voorhees fb4d0d470f whisper : fix uninitialized exp_n_audio_ctx 2023-02-21 18:58:08 +02:00
Georgi Gerganov 0d229163bb
whisper : add API for applying custom logits filters during decoding 2023-02-19 18:35:01 +02:00
Georgi Gerganov a94897bcde
whisper : by default disable non-speech tokens suppression (#473) 
This seems to be causing hallucinations in the end of the audio, e.g.:

"Thank you for listening"
"Amen"
..
 2023-02-15 21:48:49 +02:00
shikokuchuo 0336161b7d
whisper : fix signedness compiler warning (#506) 2023-02-15 19:08:25 +02:00
shibukazu cfc06bf8df
whisper : suppress non-speech-related token outputs (#473) 
* add non-speech-token suppression

* add suppress non-speech_tokens param
 2023-02-08 09:05:34 +02:00
sandrohanea 2bfe0ebc0f
whisper : fixed Beam Search Strategy and exposed whisper_pcm_to_mel_phase_vocoder (#474) 
Co-authored-by: Sandro Hanea <sandrohanea@microsoft.com>
 2023-02-08 09:01:47 +02:00
boolemancer 4dd7119deb
whisper : only trim if split_on_word is true (#476) 2023-02-08 08:43:23 +02:00
kamranjon a1c1583cc7
whisper : add whisper_full_lang_id() for getting the context lang (#461) 2023-02-05 14:46:26 +02:00
Matija Pevec d012b5c7e4
whisper : add "split_on_word" flag when using using "max_len" option (#455) 
* Update whisper.cpp

* fix: trim function

* feat: added flag to split on word

* fix: arguments for main
 2023-02-05 14:44:23 +02:00
Georgi Gerganov f3ee4a9673
whisper : reduce memory usage during inference (#431) 
* ggml : add "scratch" buffer support

* ggml : support for scratch ring-buffer

* ggml : bug fix in ggml_repeat()

* ggml : error on scratch buffer overflow

* whisper : use scratch buffers during inference (base model only)

* whisper : update memory usage for all models

* whisper : fix encoder memory usage

* whisper : use whisper_context functions instead of macros

* whisper : fix FF + remove it from README

* ggml : reuse ggml_new_i32

* ggml : refactor the scratch buffer storage

* whisper : reorder scratch buffers in the decoder

* main : add option to disable temp fallback

* Update README.md
 2023-02-04 09:45:52 +02:00
Georgi Gerganov 291980369c
whisper : suppress task tokens (#442) 2023-02-04 09:03:14 +02:00
Georgi Gerganov b992f3709e
whisper : do not provide past prompt when n_max_text_ctx == 0 2023-01-25 20:01:00 +02:00
Georgi Gerganov b5ddb16ec7
whisper : condition timestamps to be monotonically increasing (#425) 2023-01-23 20:48:26 +02:00
fitzsim ae16c21e9c
whisper : PPC64 big-endian support (#398) 
* ggml : set cache line size to 128 on POWER9

* whisper : add PPC64 big endian support
 2023-01-23 20:48:10 +02:00
Georgi Gerganov 78f166174f
whisper : fix condition for providing past prompt (critical) 
This bug has been present since v1.1.0.

Effectively, the past transcribed text wasn't being used for following
transcriptions, which likely significantly reduces the transcription
quality.

Likely related to #419
 2023-01-22 10:47:01 +02:00
Georgi Gerganov 21c569ba4a
whisper : extend information in whisper_print_timings() 2023-01-19 18:50:33 +02:00
Georgi Gerganov 1a91c19af9
whisper : perform entropy check only when we have at least 32 tokens (#412) 2023-01-18 22:52:18 +02:00
Georgi Gerganov a6cf6f4c4a
bench : minor fixes 2023-01-18 21:40:10 +02:00
Georgi Gerganov 1ccb8a46a5
bench : fix Windows linkage by moving ggml benches in whisper lib .. 2023-01-18 21:19:50 +02:00
Georgi Gerganov 8088a977af
whisper : fix possible uninitialized variables (#291) 2023-01-16 21:44:40 +02:00
Georgi Gerganov 00ea21668b
whisper : account speed_up flag for short audio (close #405) 2023-01-15 12:42:15 +02:00
Georgi Gerganov 8de452c18b
Improve decoding (#291) 
* whisper : prepare infra for new decoding strategies

* whisper : apply logit filters and compute logprobs

* whisper : add whisper_get_logits()

* whisper : separate self and cross attention memory

Initial step needed for supporting parallel decoders

* whisper : move probs_id buffer to whisper_context

* whisper : refactor kv cache into separate struct

* whisper : move self-attention kv cache to whisper_decoder

* whisper : wip decoding parameters + strategies

* whisper : wip decoding parameters + strategies (part 2)

* whisper : wip decoding parameters + strategies (part 3)

* whisper : wip decoding parameters + strategies (part 4)

* whisper : fix prompt_past update to not include prompt_init

* whisper : temperature + best_of support

* whisper : support for compression_ration_threshold

We actually use entropy, but it is similar

* command : fix example to use logits instead of obsolete probs

* whisper : handle empty sequence ranking

* whisper : add WHISPER_DEBUG + diagnostic prints + new main args

* whisper : minor fixes

* whisper : add beam-search support

* whisper : bug fix when there no previous context

* whisper : add comments

* stream : disable temperature fallback

For real-time processing, we always want a single decoder running at T=0

* whisper.swiftui : update example - fix paths + add empty folders
 2023-01-15 11:29:57 +02:00
Georgi Gerganov 4ef3398e8f
ggml : remove obsolete zeroing + comment fixes (#390) 2023-01-08 20:21:03 +02:00
boolemancer 08dc705a69
whisper : fix sample_to_timestamp calculation with 64 bit precision to avoid overflow (#388) 
* Do calculation with 64 bit precision to avoid overflow

* Update whisper.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-01-08 15:08:45 +02:00
Syahmi Azhar 1512545149
whisper : add loader class to allow loading from buffer and others (#353) 
* whisper : add loader to allow loading from other than file

* whisper : rename whisper_init to whisper_init_from_file

* whisper : add whisper_init_from_buffer

* android : Delete local.properties

* android : load models directly from assets

* whisper : adding <stddef.h> needed for size_t + code style

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-01-08 13:03:33 +02:00
Georgi Gerganov 65fdcbbbbb
whisper : revert accidental MB change 2023-01-07 16:18:21 +02:00
Georgi Gerganov d61d55cd4b
ggml : speed-up soft max via Accelerate + unroll 2023-01-07 16:16:42 +02:00
Abitofevrything a62170c656
ggml : add SSE3 and fp16 conversion lookup table (#368) 
* Improves WASM performance:
  On MacBook M1 Pro, I observe 25% faster using Firefox and 35% faster using Chrome

* Add support for SSE3 SIMD

* Add SSE3 to system information

* Add Imath support for fp16-fp32 conversions

* Add Imath to system information

* Wrap Imath calls to avoid static function warnings

* Drop Imath; Add lookup table for f16 -> f32 conversions

* Remove TODO comments

* Update SSE3 to new macro arguments

* Correct updated macro definitions

* Prefer static inline where possible

* ggml : static inlines + add public f16 <-> f32 conversions

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-01-06 18:45:59 +02:00
Thomas Fitzsimmons 1944e7c33e whisper : document POWER VSX support 2023-01-05 23:53:00 +02:00
Georgi Gerganov ad2a4ffa03
whisper : do not use F16 tensors when in F32 mode (#369) 2023-01-05 22:56:25 +02:00
Andy Maloney dd6d582977 whisper : use ranged-based for loops for readability 2023-01-05 21:20:44 +02:00
Georgi Gerganov d51c5eb906
ggml : define MIN / MAX only if not defined (minor) 2023-01-05 21:16:52 +02:00
Georgi Gerganov d97e6005e9
whisper : add whisper_n_audio_ctx and check for invalid audio_ctx 
closes #344
 2022-12-31 09:57:19 +02:00
Georgi Gerganov 68daf6e487
whisper : avoid some memory allocations 2022-12-30 13:43:48 +02:00
Georgi Gerganov ac521a566e
ggml : simplify the SIMD code (#324) 
* ggml : simplify the SIMD code

* ggml : generic reduce for all register sizes + comments
 2022-12-24 10:22:28 +02:00