Superminaren/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-24 13:37:01 +02:00
Code Issues Packages Projects Releases Wiki Activity
1,412 Commits 807 Branches 7,572 Tags
speculative-tree
Commit Graph
3 Commits
Author SHA1 Message Date
Georgi Gerganov 360a333145 common : add llama_batch_add() and llama_batch_clear() helpers 2023-10-16 12:41:33 +03:00
Georgi Gerganov 4de5a2d473 speculative : add tree-based sampling support 
ggml-ci
 2023-10-14 17:54:02 +03:00
Georgi Gerganov 8c70a5ff25 batched : add bench tool (#3545) 
* batched : add bench tool

* batched : minor fix table

* batched-bench : add readme + n_kv_max is now configurable

* batched-bench : init warm-up batch

* batched-bench : pass custom set of PP, TG and PL

* batched-bench : add mmq CLI arg
 2023-10-11 21:25:33 +03:00