 Georgi Gerganov
|
85a7d8677b
|
memory : remove KV cache size padding (#16812)
* memory : remove KV cache size padding
* cont : restore padding for n_kv tensor shape
* server : use slot context size instead of training context size
* server : simplify context limit logic
|
2025-10-28 20:19:44 +02:00 |
|
 Georgi GerganovandXuan-Son Nguyen
|
d00cbea63c
|
server : host-memory prompt caching (#16391)
* minor : code style
* server : fix prompt similarity calculation
* server : initial host-memory prompt caching
* cont
* server : refactor
* cont
* cont : make the server task of the slot const
* cont : minor [no ci]
* server : cache prompts and checkpoints only for completion tasks
* server : improve prompt caching logic
* cont : fix check for number of cached prompts [no ci]
* server : improve caching logic, add -cram CLI arg
* server : print prompt mismatch info
* cont : better naming [no ci]
* server : improve prompt cache loading logic
* server : add option to debug the slot contents (#16482)
* server : add option to debug the slot contents
* Update tools/server/server.cpp
---------
Co-authored-by: Xuan-Son Nguyen <[email protected]>
* server : add option to disable prompt cache
---------
Co-authored-by: Xuan-Son Nguyen <[email protected]>
|
2025-10-09 18:54:51 +03:00 |
|
 Johannes Gäßler
|
e81b8e4b7f
|
llama: use FA + max. GPU layers by default (#15434)
* llama: use max. GPU layers by default, auto -fa
* ggml-backend: abort instead of segfault
|
2025-08-30 16:32:10 +02:00 |
|
|
Georgi Gerganov
|
d2fcd91cf9
|
server : disable context shift by default (#15416)
* server : disable context shift by default
ggml-ci
* server : make scopr of test parameters local
|
2025-08-19 16:46:37 +03:00 |
|
|
Xuan-Son Nguyen
|
6aa892ec2a
|
server : do not return error out of context (with ctx shift disabled) (#13577)
|
2025-05-16 21:50:00 +02:00 |
|
 Diego DevesaandXuan Son Nguyen
|
1d36b3670b
|
llama : move end-user examples to tools directory (#13249)
* llama : move end-user examples to tools directory
---------
Co-authored-by: Xuan Son Nguyen <[email protected]>
|
2025-05-02 20:27:13 +02:00 |
|