History

Georgi Gerganov 0e89203b51 speculative : add tree-based sampling example (#3624 ) * sampling : one sequence per sampling context ggml-ci * speculative : add tree-based sampling support ggml-ci * speculative : reuse the n_parallel CLI param * speculative : refactor sampling * examples : fix build after sampling refactoring ggml-ci * batched : fix n_seq_id * sampling : fix malloc ggml-ci * swift : fix build ggml-ci * swift : try to fix build ggml-ci * prompts : add assistant.txt * common : add llama_batch_add() and llama_batch_clear() helpers * speculative : minor refactor ggml-ci * minor : comments + rename ggml-ci * speculative : fix off-by-one for n_drafted * speculative : fix the n_drafted fix + p constants		2023-10-18 16:21:57 +03:00
..
CMakeLists.txt	llama : custom attention mask + parallel decoding + no context swaps (#3228 )	2023-09-28 19:04:36 +03:00
parallel.cpp	speculative : add tree-based sampling example (#3624 )	2023-10-18 16:21:57 +03:00
README.md	llama : custom attention mask + parallel decoding + no context swaps (#3228 )	2023-09-28 19:04:36 +03:00

llama.cpp/example/parallel

Simplified simluation for serving incoming requests in parallel