  g2mtandGeorgi Gerganov
|
94933c8c2e
|
server : implement universal assisted decoding (#12635)
* llama-server : implement universal assisted decoding
* Erase prompt tail for kv-cache
* set vocab_dft_compatible in common_speculative
* rename ctx_main to ctx_tgt
* move vocab_dft_compatible to spec struct
* clear mem_dft, remove mem
* detokenize id_last for incompatible models
* update comment
* add --spec-replace flag
* accept special tokens when translating between draft/main models
* Escape spec-replace
* clamp draft result to size to params.n_draft
* fix comment
* clean up code
* restore old example
* log common_speculative_are_compatible in speculative example
* fix
* Update common/speculative.cpp
Co-authored-by: Georgi Gerganov <[email protected]>
* Update common/speculative.cpp
Co-authored-by: Georgi Gerganov <[email protected]>
* Update common/speculative.cpp
Co-authored-by: Georgi Gerganov <[email protected]>
---------
Co-authored-by: Georgi Gerganov <[email protected]>
|
2025-07-31 14:25:23 +02:00 |
|
|
Georgi Gerganov
|
abd4d0bc4f
|
speculative : update default params (#11954)
* speculative : update default params
* speculative : do not discard the last drafted token
|
2025-02-19 13:29:42 +02:00 |
|
 Maxim Evtush
|
7b891bdc86
|
fix: typos in documentation files (#11791)
* Update ggml.c
* Update arg.cpp
* Update speculative.h
|
2025-02-10 23:21:31 +01:00 |
|
|
Georgi Gerganov
|
d9d54e498d
|
speculative : refactor and add a simpler example (#10362)
* speculative : refactor and add a simpler example
ggml-ci
* speculative : clean-up and add comments and TODOs [no ci]
* speculative : manage context in common_speculative
ggml-ci
* speculative : simplify
ggml-ci
* speculative : simplify (cont)
ggml-ci
* speculative : add --draft-min CLI arg
* speculative : minor fixup
* make : build fixes
* speculative : do not redraft previous drafts
ggml-ci
* speculative : fix the draft sampling
ggml-ci
* speculative : fix compile warning
* common : refactor args
ggml-ci
* common : change defaults [no ci]
* common : final touches
ggml-ci
|
2024-11-25 09:58:41 +02:00 |
|