haraldwolff/whisper.cpp
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
1154 commits 41 branches 23 tags 85 MiB
Commit graph

81 commits

Author SHA1 Message Date
Johannes Gäßler 53d4d0b30d
CUDA: fixed redundant value dequantization (llama/4809) 2024-01-11 21:50:00 +02:00
Konstantin Zhuravlyov 2865e4710b
ggml : use __builtin_amdgcn_sudot4 in __dp4a for gfx11 (llama/4787) 2024-01-11 21:50:00 +02:00
Erik Scholz 11b1b63b14
fix : cuda order of synchronization when setting a buffer (ggml/679) 
* fix : cuda order of synchronization when setting a buffer

* also sync before memcpy

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-01-05 17:01:59 +02:00
Finn Voorhees a3d0aa73d1
ggml : add error handling to graph_compute (#1714) 2024-01-03 15:39:43 +02:00
Georgi Gerganov 14c57952f7 cuda : simplify expression 
Co-authored-by: slaren <slarengh@gmail.com>
 2024-01-03 14:43:51 +02:00
Georgi Gerganov 6c369d6788 cuda : mark I16 and I32 ops as unsupported 
ggml-ci
 2024-01-03 14:43:51 +02:00
Johannes Gäßler a28dacec65 CUDA: fixed tensor cores not being used on RDNA3 (llama/4697) 2024-01-03 14:43:51 +02:00
Johannes Gäßler fe3a67c546 CUDA: fix tensor core logic for Pascal and HIP (llama/4682) 2024-01-03 14:43:51 +02:00
hydai b138ff2be3 cuda: fix vmm oom issue on NVIDIA AGX Orin (llama/4687) 
Signed-off-by: hydai <hydai@secondstate.io>
 2024-01-03 14:43:51 +02:00
Georgi Gerganov e77b27c331
sync : ggml (VMM, sync-ggml-am, dotprod ARM fixes, CUDA fixes) (#1691) 
* scripts : add sync-ggml-am.sh

* sync : ggml (VMM, ARM dot prod fix, etc.)

* build : fix CUDA build

* ggml : fix some mul mat cases + add tests for src1 F16

dbd02958fa
 2023-12-29 11:30:47 +02:00
Georgi Gerganov 3a5302108d
sync : ggml (ggml_scale, ggml_row_size, etc.) (#1677) 
* sync : ggml

* sync : llama.cpp

* talk-llama : fix obsolete param

* ggml-alloc : fix ggml_tallocr_is_own

* talk.wasm : update to new ggml

* ggml : fix type punning in ggml_scale

* ggml : cuda jetson + arm quants warnings
 2023-12-22 17:53:39 +02:00
Georgi Gerganov 8171e621fc
sync : ggml (Metal fixes, new ops, tests) (#1633) 
* sync : ggml (Metal fixes, new ops, tests)

* cuda : fix bin bcast when src1 and dst have different types
 2023-12-13 21:55:03 +02:00
Georgi Gerganov afce6fa113
sync : ggml (new ops, new backend, etc) (#1602) 
* sync : ggml (new ops, new backend, etc)

* whisper : remove obsolete broadcasting code

* ggml : remove backend self-registers + fix ggml_concat + n_task logic

* metal : fix assert

* metal : print resource path

* whisper : fix bug if metal init fails
 2023-12-07 22:27:19 +02:00
Georgi Gerganov 010c8ec3ab
cuda : sync some minor stuff from llama.cpp (#1548) 2023-11-24 12:36:21 +02:00
Georgi Gerganov ca23f8ee6d
cuda : assert ggml_add sources to be contiguous 2023-11-19 10:32:08 +02:00
Georgi Gerganov b6c5f49b78
whisper : add batched decoding (#1486) 
* whisper : add whisper_batch

* whisper : move kv_self to whisper_state

* whisper : full batched decoding support

* whisper : fix memory leak in whisper_batch

* whisper : fix mem leak again + remove oboslete function

* whisper : clear kv cache when using whisper_decode API

* whisper : speed-up sampling

* whisper : fix decoders initializer

* bench : add batch size 5 bench

* whisper : add comment about the KV cache size

* whisper : add check for max number of decoders

* whisper : avoid starting sampling threads with bs=1

* whisper : enable beam-search by default

* cuda : sync llama.cpp fixes
 2023-11-15 16:12:52 +02:00
Georgi Gerganov 3172006a24
ggml : fix some compile warnings 2023-11-12 16:36:20 +02:00
Georgi Gerganov b0502836b8
whisper : add full CUDA and Metal offloading (#1472) 
* whisper : migrate to ggml-backend

* whisper : fix logit reading

* whisper : fix tensor allocation during load

* whisper : fix beam-search with CUDA

* whisper : free backends + fix compile warning

* whisper : print when CUDA is enabled

* whisper : fix CoreML

* make : clean-up

* talk : fix compile warning

* whisper : support ggml_conv with CUDA and Metal (#1473)

* ggml : add CUDA support for ggml_conv

* whisper : remove ggml_repeat for conv bias + single backend

* cuda : fix im2col kernel

* metal : add im2col support + mul mat-vec f16 x f16

* bench-all : add q4 models

* whisper : clean-up

* quantize-all : fix

* ggml : im2col opts

* whisper : avoid whisper_model_data wrapper

* whisper : add note that ggml_mul_mat_pad does not work with CUDA

* whisper : factor out graph compute in common function

* whisper : fixes

* whisper : fix UB with measure buffers

* whisper : try to fix the parallel whisper_state functionality (#1479)

* whisper : try to fix the parallel whisper_state functionality

* whisper : fix multi-state Metal

* whisper : free backend instances in whisper_state
 2023-11-12 15:31:08 +02:00
Georgi Gerganov 6d4d0b5b4b
cuda : fix HIPBLAS build 2023-11-05 19:41:15 +02:00
Georgi Gerganov f96e1c5b78
sync : ggml (backend v2, k-quants, CUDA opts, Metal opts, etc.) (#1422) 
* sync : ggml (backend v2, k-quants, CUDA opts, Metal opts, etc.)

* metal : allow env metal variable to override resource path (#1415)

* Allow env variable to override resource path

* Update ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* sync : restore common / main from `master`

* sync : restore whisper from `master`

* talk-llama : update to latest llama.cpp

* ruby : fix build

* ggml : fix 32-bit ARM build

* ggml : fix MIN / MAX macro collisions + update ios bindings

* ggml : fix ifdefs and MIN / MAX again

* exampels : fix Obj-C and Swift examples

* ggml : fix 32-bit ARM compatibility

* ggml : one more attempt to fix 32-bit ARM compat

* whisper : fix support for larger graphs

---------

Co-authored-by: Chris Raethke <codesoda@users.noreply.github.com>
 2023-11-03 21:35:05 +02:00
Georgi Gerganov 80c1512fd5
sync : ggml (const correctness) 2023-09-15 14:49:56 +03:00
Georgi Gerganov bfc73f1fa2
sync : ggml (CUDA faster rope) 2023-09-08 15:01:26 +03:00
Georgi Gerganov c3f319d7c2
ggml : sync latest llama.cpp (view_src + alloc improvements) (#1247) 
* ggml : sync latest llama.cpp (view_src + alloc improvements)

* ggml : fix build
 2023-09-05 20:57:27 +03:00
Georgi Gerganov 59a3d0cb57
ggml : sync (ggml-alloc, GPU, eps, etc.) (#1220) 
* ggml : sync (ggml-alloc, GPU, eps, etc.)

* ggml : fix build

* wasm : fix build
 2023-09-05 13:54:40 +03:00
ardfork cb5fb0a12d
whisper : initial hipBLAS support (#1209) 2023-08-27 20:03:58 +03:00
Georgi Gerganov d6509bf78d
ggml : sync latest repo (mostly refactoring changes) 2023-07-02 21:46:09 +03:00
Georgi Gerganov 5feb0dffba
ggml : sync latest ggml lib 2023-06-25 14:30:44 +03:00
Georgi Gerganov e410cfc3ce
ggml : sync latest ggml repo 
- new Q4 and Q8 quantization
- updated CUDA
 2023-05-20 18:56:30 +03:00
Georgi Gerganov e693074aa6
ggml : sync latest ggml 
- New Q4 and Q5 formats
- Various improvements
 2023-05-14 18:04:23 +03:00
Georgi Gerganov 0bcb64b184
ggml : sync ggml (clBLAST + tensor names) 2023-05-02 21:24:18 +03:00
Georgi Gerganov acec73ab6e
ggml : sync latest ggml + llama.cpp updates (quantization) 2023-04-29 12:32:28 +03:00