Superminaren/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-07 20:47:30 +02:00
Code Issues Packages Projects Releases Wiki Activity
4,489 Commits 780 Branches 7,352 Tags
gg/ci-python
Commit Graph
3 Commits
Author SHA1 Message Date
Xuan Son NguyenandGeorgi Gerganov 0da5d86026 server : allow using LoRA adapters per-request (#10994) 
* slot.can_batch_with

* lora per request

* test: force disable cache prompt

* move can_batch_with check

* fix condition

* add slow test with llama 8b

* update docs

* move lora change task to queue

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <[email protected]>

* lora_base

* remove redundant check

---------

Co-authored-by: Georgi Gerganov <[email protected]>
 2025-01-02 15:05:18 +01:00
Georgi Gerganov 1da7b76569 server : fix speculative decoding with context shift (#10641) 
* server : fix speculative decoding with context shift

ggml-ci

* server : take into account speculative limits

ggml-ci

* server : add tests
 2024-12-04 22:38:20 +02:00
Xuan Son Nguyen b782e5c7d4 server : add more test cases (#10569) 
* server : add split model test

* add test speculative

* add invalid cases
 2024-11-29 21:48:56 +01:00