Georgi Gerganov
|
8c70a5ff25
|
batched : add bench tool (#3545)
* batched : add bench tool
* batched : minor fix table
* batched-bench : add readme + n_kv_max is now configurable
* batched-bench : init warm-up batch
* batched-bench : pass custom set of PP, TG and PL
* batched-bench : add mmq CLI arg
|
2023-10-11 21:25:33 +03:00 |
|