Superminaren/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-07 20:47:30 +02:00
Code Issues Packages Projects Releases Wiki Activity
1,286 Commits 780 Branches 7,352 Tags
eed3fd4234ea055493657ea58b4ad14e6797922f
Commit Graph
11 Commits
Author SHA1 Message Date
Georgi Gerganov eed3fd4234 parallel : count cache misses 2023-09-19 23:47:47 +03:00
Georgi Gerganov 8a9aca37c1 parallel : remove question with short answers 2023-09-19 23:34:30 +03:00
Georgi Gerganov 4b5f3cd6bf parallel : process system prompt once + configurable paramters + llama API 2023-09-19 17:00:42 +03:00
Georgi Gerganov 82e20e9ba0 parallel : remove new line from prompt 2023-09-19 13:54:41 +03:00
Georgi Gerganov 16090a5dde parallel : fix sequence termination criteria 2023-09-19 13:29:29 +03:00
Georgi Gerganov 806d397c1a parallel : try smaller batches when the KV cache is fragmented 2023-09-19 13:21:36 +03:00
Georgi Gerganov 36714e16d0 parallel : various improvements 2023-09-19 12:29:37 +03:00
Georgi Gerganov fa0e677820 llama : extend batch API to select which logits to output 2023-09-19 00:24:13 +03:00
Georgi Gerganov 897caccdf4 fixes : speculative KV cache + llama worst-case graph 2023-09-18 22:32:28 +03:00
Georgi Gerganov 466b513851 parallel : disable hot-plug to avoid cache fragmentation 2023-09-18 21:34:20 +03:00
Georgi Gerganov 0161372b9a parallel : example for serving multiple users in parallel 2023-09-18 20:37:28 +03:00