haraldwolff/llama.cpp
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
llama.cpp/examples
History
Johannes Gäßler 905d87b70a
ggml : GPU-accelerated token generation (#1412) 
* CUDA kernel for q4_0 dequant. + mat. vec. mult.

* Added q4_1 via template

* Added missing __syncthreads();

* --gpu_layers -> --gpu-layers

* Shorter dequantize_mul_mat_vec line

* q5_0 dequantize_mul_mat kernel

* More readable dequantize_mul_mat_vec logic

* dequantize_mul_mat_vec kernels for q5_1, q8_0, f16

* llama : offload "output" tensor to GPU too + coding style fixes

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-05-13 16:38:36 +03:00
..
baby-llama ggml : implement backward pass for llama + small training-llama-from-scratch example (#1360) 2023-05-13 15:56:40 +03:00
benchmark llama : fix compile warnings 2023-05-02 23:09:08 +03:00
embedding embedding : remove unused code (#1426) 2023-05-13 10:24:20 +03:00
jeopardy examples : add Jeopardy example (#1168) 2023-04-28 19:13:33 +03:00
main llama : fix --mtest option (close #1414) 2023-05-12 21:44:20 +03:00
perplexity llama : require first token to be BOS (#1303) 2023-05-08 17:41:54 +03:00
quantize ggml : remove bit shuffling (#1405) 2023-05-12 00:23:08 +03:00
quantize-stats Add git-based build information for better issue tracking (#1232) 2023-05-01 18:23:47 +02:00
save-load-state Add git-based build information for better issue tracking (#1232) 2023-05-01 18:23:47 +02:00
alpaca.sh examples : Improve Alpaca Default Repeat Penalty: Better Match Alpaca.cpp Experience (#1107) 2023-04-22 09:54:33 +03:00
chat-13B.bat Create chat-13B.bat (#592) 2023-03-29 20:21:09 +03:00
chat-13B.sh examples : read chat prompts from a template file (#1196) 2023-05-03 20:58:11 +03:00
chat.sh If n_predict == -1, generate forever 2023-03-25 21:51:41 +02:00
CMakeLists.txt ggml : implement backward pass for llama + small training-llama-from-scratch example (#1360) 2023-05-13 15:56:40 +03:00
common.cpp ggml : GPU-accelerated token generation (#1412) 2023-05-13 16:38:36 +03:00
common.h ggml : GPU-accelerated token generation (#1412) 2023-05-13 16:38:36 +03:00
gpt4all.sh examples : add -n to alpaca and gpt4all scripts (#706) 2023-04-13 16:03:39 +03:00
Miku.sh examples : various prompt and example fixes (#1298) 2023-05-03 18:26:47 +03:00
reason-act.sh add example of re-act pattern (#583) 2023-03-29 10:10:24 -05:00