0cc4m
|
9c399689ec
|
Vulkan Improvements (llama/5835)
* Improve dequant shaders, add fast q4_0 dequant
* Optimize dmmv non-kquants for GCN
Remove unnecessary SPIR-V shader duplication
* Fix q4_0 dequant dispatch sizes
Fix backend free bug
* Optimize dequant shaders for q4_1, q5_0, q5_1 and q8_0
* Add unary and binary op shader templates
* Fix Vulkan check results
* Enable non-contiguous support for simple ops
* Add argsort
Basic q4_0 mmq shader and unit test
* Speed up q4_0 dequant code, enable mmq for q4_0
* Rework matmul pipeline selection
* Add soft_max alibi support
* Add q4_1, q5_0, q5_1 and q8_0 dequant mat mat mul shaders
* Add environment variable GGML_VK_FORCE_MAX_ALLOCATION_SIZE to limit max buffer size
Rename GGML_VULKAN_DISABLE_F16 to GGML_VK_DISABLE_F16 for consistency
|
2024-03-08 11:38:33 +02:00 |
Georgi Gerganov