206fc93396
whisper.wasm : add small and small.en models
2023-01-18 21:58:55 +02:00
472a473fd1
main : add an option to accept optional output filenames ( #424 )
...
* Add an option to accept optional output filenames
* Format the file
Co-authored-by: Chia-Hsiang Cheng <gary.chiahsiang.cheng@gmail.com>
2023-01-18 21:26:31 +02:00
9ba66c2fad
stream : fix handling of --step == --length ( #416 )
2023-01-18 21:22:52 +02:00
1ccb8a46a5
bench : fix Windows linkage by moving ggml benches in whisper lib ..
2023-01-18 21:19:50 +02:00
1290fc6457
bench : add memcpy and ggml_mul_mat benchmarks
2023-01-18 20:31:46 +02:00
49b529ba74
whisper.android : add support for loading directly from asset in C ( #415 )
2023-01-16 21:57:35 +02:00
c9aeb33676
stream : fix --keep_context argument to be used correctly ( #354 )
2023-01-16 19:37:40 +02:00
c3991bbb24
Update README.md
2023-01-15 14:08:12 +02:00
fafd78945d
bench.wasm : print system info
2023-01-15 11:34:03 +02:00
8de452c18b
Improve decoding ( #291 )
...
* whisper : prepare infra for new decoding strategies
* whisper : apply logit filters and compute logprobs
* whisper : add whisper_get_logits()
* whisper : separate self and cross attention memory
Initial step needed for supporting parallel decoders
* whisper : move probs_id buffer to whisper_context
* whisper : refactor kv cache into separate struct
* whisper : move self-attention kv cache to whisper_decoder
* whisper : wip decoding parameters + strategies
* whisper : wip decoding parameters + strategies (part 2)
* whisper : wip decoding parameters + strategies (part 3)
* whisper : wip decoding parameters + strategies (part 4)
* whisper : fix prompt_past update to not include prompt_init
* whisper : temperature + best_of support
* whisper : support for compression_ration_threshold
We actually use entropy, but it is similar
* command : fix example to use logits instead of obsolete probs
* whisper : handle empty sequence ranking
* whisper : add WHISPER_DEBUG + diagnostic prints + new main args
* whisper : minor fixes
* whisper : add beam-search support
* whisper : bug fix when there no previous context
* whisper : add comments
* stream : disable temperature fallback
For real-time processing, we always want a single decoder running at T=0
* whisper.swiftui : update example - fix paths + add empty folders
2023-01-15 11:29:57 +02:00
a6dbd9188b
stream : fix a bug that inserted a lot of empty audio at the start
...
The quality was terrible due to this
2023-01-14 19:20:47 +02:00
1512545149
whisper : add loader class to allow loading from buffer and others ( #353 )
...
* whisper : add loader to allow loading from other than file
* whisper : rename whisper_init to whisper_init_from_file
* whisper : add whisper_init_from_buffer
* android : Delete local.properties
* android : load models directly from assets
* whisper : adding <stddef.h> needed for size_t + code style
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-01-08 13:03:33 +02:00
52a3e0c92a
ggml : improve vec_dot_f16 unrolling in flash_attn_f16
2023-01-08 11:41:18 +02:00
d1ea1220ff
command : clean-up / refactoring / formatting ( #383 )
2023-01-07 21:43:24 +02:00
9c4a1522f6
command : always-prompt mode ( #383 )
2023-01-07 21:41:11 +02:00
87dd4a3081
talk.wasm : bump memory usage + update whisper.js
2023-01-06 21:13:44 +02:00
6b351bb669
command : add "guided-mode" video demo in the README.md
2023-01-06 18:59:26 +02:00
b3c865083e
ci : add emscripten build
2023-01-05 22:10:20 +02:00
a0d4f8e65c
main : make whisper_print_segment_callback() more readable ( close #371 )
2023-01-05 21:45:05 +02:00
196d738974
minor : close #370 + Makefile build info print change
2023-01-05 21:35:45 +02:00
84c6b42e65
cmake : update to 3.19 ( #351 )
...
- update from 3.0 (from 2014) to 3.19 (from 2020)
- move some global setting onto the targets (through a cmake include)
2023-01-05 21:22:48 +02:00
a466c3404d
stream : fix data race on bool + avoid division-by-zero
2023-01-02 10:20:50 +02:00
f00509d57c
command : refactor to split command list & general transcription modes ( #331 )
...
This makes it easier to understand if you're looking for only one of the capabilities.
2022-12-31 14:08:57 +02:00
a593b932e4
main : add -ocsv, aka --output-csv to output a CSV file
...
Adds -ocsv, aka --output-csv feature to examples/main, which outputs a CSV file containing lines formatted as follows <startTime-in-integer-milliseconds>, <endTime-in-integer-milliseconds>, "<transcript-line-including-commas>".
2022-12-29 14:04:00 +02:00
331c0bbddc
examples : fix memory leak on failure to load gpt2 model ( #323 )
2022-12-23 20:19:07 +02:00
dc90efd504
examples : small code cleanups ( #322 )
...
- remove unnecessary initialization of string to ""
- use empty() instead of checking size()
- use emplace_back instead of push_back
- use nullptr instead of NULL
- remove unnecessary call to .data() on string
- use character overload of find_first_of() instead of passing a string
2022-12-23 20:18:51 +02:00
0f4227d9ee
examples : add whisper.swiftui demo app ( #308 )
...
* Add SwiftUI demo project.
* Add -DGGML_USE_ACCELERATE
2022-12-23 10:56:18 +02:00
91fc08c641
Build a vfpv4 library for armeabi-v7a and do runtime detection to select the right library
2022-12-22 16:47:54 +02:00
e1432dd91a
Check for both __ARM_NEON and __ARM_FEATURE_FMA so that the project can be compiled for armv7a.
...
Android armeabi-v7a's NEON support doesn't support FMA unless configured with `-mfpu=neon-fp-armv8`, which would need runtime checks.
* Also removed ABI filter from Android project.
2022-12-22 16:47:54 +02:00
22193cbfe8
Bump NDK version
2022-12-22 16:47:54 +02:00
90564f85f9
Update README.md
2022-12-19 22:09:21 +02:00
99da1e5cc8
cmake : enable and fix -Wall -Wextra -Wpedantic C++ warnings
2022-12-19 20:45:08 +02:00
8e3f129b4d
minor : resolves some of warnings when compiling with clang/clang++ ( #294 )
...
* Resolves some of warnings when compiling with clang/clang++
Mostly nit stuff that clang catches when compiling with -Wall -Wextra
-pedantic.
- Fix comparison between sign/unsigned integers.
- Passes a constant reference (const&) instead of copying each time.
* minor : normalize coding style
* minor : fix warning
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2022-12-19 20:19:01 +02:00
fba10a4c68
whisper : language auto-detect ( #59 )
2022-12-17 18:49:44 +02:00
32fbc8cd04
main : add option to print the progress ( #276 )
2022-12-16 20:20:43 +02:00
b8065d90f5
main : add "--prompt" command line argument ( #90 )
...
This allows to provide an initial prompt to be used at the start of the
processing.
2022-12-16 19:43:16 +02:00
4312995974
command : better indentation
2022-12-16 19:38:18 +02:00
5eeeb3412d
command : update README, show how to use guided mode
2022-12-16 19:38:18 +02:00
6a69e3ae27
command : adding guided mode
2022-12-16 19:38:18 +02:00
ea19ed33f1
Update README.md ( #46 )
...
Add references to the new Android app
2022-12-16 19:28:51 +02:00
675e787171
Add Android sample ( #277 )
...
* Add Android sample
* Use main project C files
* Stop existing playback before starting new playback
* Make text scrollable
* Stop playback when starting to record
* Remove extra var
2022-12-16 19:20:13 +02:00
a82d331034
stream : update README.md + comments
2022-12-16 18:04:19 +02:00
5a5c5ddcca
Update README.md
2022-12-15 20:38:08 +02:00
34e0b4b9ef
stream : fix build
2022-12-15 20:15:36 +02:00
b0f8013eb9
stream : add sliding window mode
2022-12-15 19:59:17 +02:00
a613f16aec
talk : improve prompting
2022-12-12 23:44:36 +02:00
f309f97df6
Node.js package ( #260 )
...
* npm : preparing infra for node package
* npm : package infra ready
* npm : initial version ready
* npm : change name to whisper.cpp
whisper.js is taken
2022-12-12 20:17:27 +02:00
aa6adda26e
talk : make compatible with c++11 (part 2)
2022-12-11 20:34:04 +02:00
444349f4ec
talk : make compatible with c++11
2022-12-11 20:19:17 +02:00
6ed786957e
Add newline per segment for text output ( #254 )
2022-12-11 20:00:29 +02:00
fcf515de60
bench.wasm : same as "bench" but runs in the browser ( #89 )
2022-12-11 11:09:10 +02:00
85c9ac18b5
Update README.md
2022-12-10 16:54:57 +02:00
b7c85d1ea6
talk : fix build for MSVC
2022-12-10 16:51:58 +02:00
3b1aacbe6d
talk : talk with AI in the terminal
2022-12-10 16:51:58 +02:00
56822621a8
twitch.sh : various fixes and polishing
...
- check if streamlink is installed
- fix audio chunking
- change default threads to 4
2022-12-08 19:20:04 +02:00
9e5f3ddc16
Allow for Twitch.tv live transcription
...
We rely on streamlink library to give us a stream, then we proceed similarly to
the radio livestream example.
2022-12-08 19:20:04 +02:00
47afb93c3c
yt-wsp.sh : improve usage instructions
2022-12-07 22:12:08 +02:00
575c53dc41
yt-wsp.sh : fix usage instruction + comment
2022-12-07 21:12:55 +02:00
faa85f9840
livestream.sh : remove obsolete comment
2022-12-07 04:41:43 +02:00
9fe7306f4b
models : add the new "large" model release by OpenAI
...
The old "large" model is now renamed "large-v1".
If you have been using it, make sure to rename it and download the new
"large" model for best results.
2022-12-06 18:48:57 +02:00
57e0e6b700
livestream : handle ffmpeg errors gracefully and stabilize transcript
2022-12-01 20:49:09 +02:00
4f7363077f
livestream : minor changes
2022-12-01 19:47:58 +02:00
093c840dee
livestream : fix losing words across audio chunk ( #195 )
...
* improve livestream script
* Update examples/livestream.sh
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Paul Edwards <paul.edwards@semiformal.net>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2022-12-01 19:18:22 +02:00
4698dcdb52
whisper : add mechanism for aborting the whisper_full() computation
2022-11-27 20:42:45 +02:00
164df0d447
whisper.objc : fix context + broken readme links
2022-11-27 10:52:27 +02:00
e266cb0723
whisper.objc : add real-time processing ( #97 )
...
Similar to the "stream" app
2022-11-26 18:32:46 +02:00
c207eed431
whisper.objc : fix build warnings
2022-11-26 16:27:04 +02:00
a425365b82
yt-wsp.sh : script to easily transcribe VODs
...
Thanks to @DaniruKun
ref: https://gist.github.com/DaniruKun/96f763ec1a037cc92fe1a059b643b818
Usage:
cd whisper.cpp
make
./examples/yt-wsp.sh <video-url>
2022-11-26 12:54:42 +02:00
68ecadbbc9
command.wasm : add voice assistant example for the Web ( #171 )
...
Same as the command-line tool "command", but runs in the browser
Also, added helper script "extra/deploy-wasm.sh" and fixed some timing
constants for the WASM examples.
2022-11-26 11:40:06 +02:00
c536ff4005
minor : add comment for using "generate_karaoke.sh"
2022-11-26 10:22:42 +02:00
cb70b07db5
livestream.sh : simple tool to transcribe audio livestreams ( #185 )
2022-11-26 10:05:37 +02:00
3c390ffe38
stream.wasm : add web-based real-time transcription ( #112 )
2022-11-25 23:57:46 +02:00
be16dfa038
whisper.wasm : do not block page while processing ( close #86 )
2022-11-25 23:07:42 +02:00
0f619b52ce
main : add stereo-channel-based diarization ( #64 )
...
Not tested - I don't have stereo dialog audio
2022-11-25 22:08:58 +02:00
1246dd023e
command : add demonstration video
2022-11-25 20:23:58 +02:00
0be27bbd92
command : fix build + fix README + add bold printing
2022-11-25 19:53:50 +02:00
bc88eb13c6
examples : add "command" tool ( #171 )
2022-11-25 19:36:57 +02:00
b8ce25dec1
refactoring : more readable code
2022-11-25 19:28:04 +02:00
e4805d9601
wasm : refactor wasm example + reuse fetch mechanism
2022-11-24 23:13:26 +02:00
ff36415a86
talk.wasm : update video link + some minor fixes
2022-11-24 20:15:24 +02:00
025ff465b6
Update README.md
...
Use a less cringy video to demo talk.wasm lol
2022-11-24 20:09:45 +02:00
abce28ea99
talk.wasm : move to https://whisper.ggerganov.com/talk
...
This way, we can share the same models across different WASM examples
and not have to download them for each page
2022-11-24 18:24:06 +02:00
454b91de16
main : fix dangling pointer when using stdin for input ( #65 )
2022-11-24 17:53:51 +02:00
d7024cf9dc
main, stream : remove --verbose flag ( #178 )
2022-11-24 17:52:04 +02:00
37422ed733
talk.wasm : add audio pre-processing + bump memory
2022-11-24 00:34:00 +02:00
be3b720f96
talk.wasm : refactoring + update README.md
2022-11-24 00:08:57 +02:00
49706a658a
minor : updates few prints + fix buttons in whisper.wasm
2022-11-23 17:19:21 +02:00
e5dcdabbb8
unicode : fix character replacement (thanks to @tamo)
2022-11-23 08:24:29 +02:00
dad109c3f1
close #109 : add fetching of the model over HTTP (whisper.wasm)
2022-11-22 22:48:56 +02:00
326573de9a
talk.wasm : final touches
2022-11-22 22:22:17 +02:00
9aea96f774
talk.wasm : polishing + adding many AI personalities
2022-11-22 20:10:20 +02:00
385236d1d3
stream : "-kc" now enables context keeping from previous segment ( #90 )
...
By default, the context keeping is disabled
2022-11-22 18:21:15 +02:00
63ae03b8e0
Prompt previous tokens for streaming ( #163 )
...
* feat: prompt previous tokens for streaming
I used a vector pointer instead of vector itself because it gave weird errors, and why not
* convert vector to use with C api
* feat: remove old refs, check for prompt size
* feat: use better way of getting the pointer
2022-11-22 18:10:35 +02:00
78116f8eda
talk.wasm : update README.md
2022-11-21 22:42:29 +02:00
a4dfbeecf9
talk.wasm : GPT-2 meets Whisper in WebAssembly ( #155 )
...
* talk : initial real-time transcription in the browser
* talk : polishing the UI
* talk : ready for beta testing
* talk.wasm : rename example
2022-11-21 22:20:42 +02:00
f2df9bd768
stream : add "max_tokens" cli arg
...
Controls the max tokens per segment for the stream example
2022-11-20 21:22:41 +02:00
fb8d77f760
stream : add "audio_ctx" parameter
...
Used to overwrite the audio context size of the Encoder.
For example, setting "audio_ctx = 512" will make it run about 3 times
faster, processing about 10s of audio, instead of 30s.
The transcription quality drops, but this can be used for real-time
streaming purposes where performance is important.
2022-11-20 21:22:41 +02:00
62b5ff875c
stream : add "max_tokens" parameter
...
Used to limit the number of tokens in a segment.
Useful to battle with word repetition when using partial encoder context
2022-11-20 21:22:41 +02:00
d351771a4b
stream : add "single_segment" option
...
Force the entire audio chunk to be transcribed into a single segment
2022-11-20 21:22:41 +02:00
c058aaf22e
stream : partial encoder experiments
2022-11-20 21:22:41 +02:00
83c742f1a7
whisper : add option to speed up the audio tempo by x2
...
Using a Phase Vocoder for speeding up the audio tempo by scaling down
the frequencies in the frequency domain.
This reduces the computation in the Encoder by a factor of 2.
The transcription accuracy is degraded, but for slow to normal speech -
it seems to be still very good.
I think this can find application for real-time transcription - i.e. the
"stream" example.
2022-11-13 16:25:43 +02:00
7519eabf65
Adds support for stdin wav input
2022-11-09 20:37:23 +02:00
c30bffc8a5
ref #22 : add "duration" option
...
Can be used to partially process a recording
2022-11-07 20:14:52 +02:00
c71363f14c
examples : add simple script for generating Karaoke video
2022-11-06 09:22:50 +02:00
d42cf6d0df
Update README.md
2022-11-04 22:26:08 +02:00
ef47d77492
main : fix generated bash script
2022-11-04 18:30:38 +02:00
d5afebd37c
whisper : token-level timestamp refactoring ( #49 , #120 )
...
This turned out pretty good overall. The algorithm has been moved from
main.cpp to whisper.cpp and can be reused for all subtitles types. This
means that now you can specify the maximum length of the generated
lines. Simply provide the "-ml" argument specifying the max length in
number of characters
2022-11-02 21:45:54 +02:00
6fb98370ba
main : add some comments for the word-level timestamp algorithm
2022-11-01 22:35:21 +02:00
0729da9a3b
main : fix some edge cases for word-level timestamps
2022-11-01 22:09:25 +02:00
5dc74e3aff
Update README.md
2022-10-31 22:06:05 +02:00
ac8ef34039
Update README.md
2022-10-31 20:19:41 +02:00
dc12994603
Update README.md
2022-10-30 17:11:37 +02:00
57fb46f307
main : add option for word-leve timestamps (very experimental)
2022-10-30 17:06:57 +02:00
5a9e4260a6
stream : add "--capture" option to select capture device (ref #10 )
2022-10-30 08:27:04 +02:00
12fb303d9d
whisper.wasm : update system info print
2022-10-29 20:32:41 +03:00
2827cbbbe8
main : merge parallel example in main
2022-10-29 19:37:19 +03:00
0b2dc3c82c
parallel : working
2022-10-29 19:37:19 +03:00
85d6e1e1e7
main : fix sampling time + add max_context parameter
2022-10-29 19:37:19 +03:00
72e9cdd6bf
parallel : adding tool for parallel transformer inference
2022-10-29 19:37:19 +03:00
b89f8960ca
Update README.md
2022-10-28 21:40:52 +03:00
6f82320b05
Create README.md
2022-10-28 20:25:37 +03:00
2298310dd8
whisper.nvim : add helper script for the Neovim integration
2022-10-28 20:25:37 +03:00
8347a7bb6a
stream : few updates to make it compatible for Vim usage ( #99 )
2022-10-27 22:10:50 +03:00
ebb01b9e33
Print system info at start of program
2022-10-27 17:22:19 +03:00
2400660f3f
Print system info in main
2022-10-26 22:54:09 +03:00
a6c786d5dc
Update README.md
2022-10-25 20:53:48 +03:00
91dcf5f35b
Update README.md
2022-10-25 20:53:48 +03:00
113a4f06d8
Update README.md
2022-10-25 20:53:48 +03:00
47e78b7288
Update README.md
2022-10-25 20:53:48 +03:00
34bb3ab0cf
ggml : add system info functions
2022-10-25 20:53:48 +03:00
c6710efde2
refactoring : move main + stream in examples + other stuff
2022-10-25 20:53:48 +03:00
d4f94ce427
Update README.md
2022-10-24 18:23:07 +03:00
a52ee08c1e
objc : polishing the sample application
2022-10-24 18:23:07 +03:00
b41f4a90eb
Create README.md
2022-10-24 18:23:07 +03:00
bb1ee266d2
ios : whisper.objc example
2022-10-24 18:23:07 +03:00
3e69a6071d
Update README.md
2022-10-23 08:04:33 +03:00
f4aa01c2f8
Update README.md
2022-10-22 19:30:35 +03:00
6b45e37b2b
Update README.md and finalize the whisper.wasm example
2022-10-22 18:54:01 +03:00
491ecd7056
wip : polishing WASM example
2022-10-22 18:54:01 +03:00
e905c6f827
wip : initial WASM port
...
Works but it is very slow because no SIMD is used.
For example, jfk.wav is processed in ~23 seconds using "tiny.en" model
2022-10-22 18:54:01 +03:00
Georgi Gerganov
Chia-Hsiang Cheng
Digipom
Syahmi Azhar
David
Andy Maloney
Niels Mayer
Matheus de Sousa
Lexevolution
keyehzy
semiformal-net
M. Eren Akbiyik
Alan