 Georgi Gerganov
|
dfde5993ea
|
common : add common_speculative_is_compat() (#19270)
* llama : add llama_memory_can_rm_suffix()
* Revert "llama : add llama_memory_can_rm_suffix()"
This reverts commit d30e59b62a.
* spec : check if the target context is compatible for spec decoding
|
2026-02-06 16:47:22 +02:00 |
|
|
Georgi Gerganov
|
dabaa2e77a
|
spec : add ngram-mod (#19164)
* spec : add ngram-mod
* cont : simplify + keep track of occupancy
* cont : cleanup
* cont : move initialization to common/speculative
* cont : cleanup
* cont : cleanup
* cont : fix
|
2026-01-30 18:21:48 +02:00 |
|
 
|
72d3b1898a
|
spec : add self‑speculative decoding (no draft model required) + refactor (#18471)
* server: introduce self-speculative decoding
* server: moved self-call into speculative.cpp
* can_speculate() includes self-speculation
Co-authored-by: Georgi Gerganov <[email protected]>
* server: can_speculate() tests self-spec
* server: replace can_speculate() with slot.can_speculate()
Co-authored-by: Sigbjørn Skjæret <[email protected]>
* common: use %zu format specifier for size_t in logging
Co-authored-by: Sigbjørn Skjæret <[email protected]>
* server: can_speculate() requires a task instance
* common: ngram map, config self-speculative decoding
* common: add enum common_speculative_type
* common: add vector of speculative states
* common: add option --spec-draftless
* server: cleanup (remove slot.batch_spec, rename)
* common: moved self-spec impl to ngram-map
* common: cleanup (use common_speculative_state_draft)
* spec : refactor
* cont : naming
* spec: remove --spec-config
* doc: (draftless) speculative decoding
* common: print performance in spec decoding
* minor : cleanup
* common : better names
* minor : cleanup + fix build
* minor: comments
* CODEOWNERS: add common/ngram-map.* (#18471)
* common : rename speculative.draftless_type -> speculative.type
* ngram-map : fix uninitialized values
* ngram-map : take into account the input can become shorter
* ngram-map : revert len check for now
* arg : change `--spec-draftless` -> `--spec-type`
* spec : add common_speculative_state::accept()
* spec : refactor + add common_speculative_begin()
* spec : fix begin() call with mtmd
* spec : additional refactor + remove common_speculative_params
---------
Co-authored-by: Georgi Gerganov <[email protected]>
Co-authored-by: Sigbjørn Skjæret <[email protected]>
|
2026-01-28 19:42:42 +02:00 |
|
 g2mtandGeorgi Gerganov
|
94933c8c2e
|
server : implement universal assisted decoding (#12635)
* llama-server : implement universal assisted decoding
* Erase prompt tail for kv-cache
* set vocab_dft_compatible in common_speculative
* rename ctx_main to ctx_tgt
* move vocab_dft_compatible to spec struct
* clear mem_dft, remove mem
* detokenize id_last for incompatible models
* update comment
* add --spec-replace flag
* accept special tokens when translating between draft/main models
* Escape spec-replace
* clamp draft result to size to params.n_draft
* fix comment
* clean up code
* restore old example
* log common_speculative_are_compatible in speculative example
* fix
* Update common/speculative.cpp
Co-authored-by: Georgi Gerganov <[email protected]>
* Update common/speculative.cpp
Co-authored-by: Georgi Gerganov <[email protected]>
* Update common/speculative.cpp
Co-authored-by: Georgi Gerganov <[email protected]>
---------
Co-authored-by: Georgi Gerganov <[email protected]>
|
2025-07-31 14:25:23 +02:00 |
|
|
Georgi Gerganov
|
abd4d0bc4f
|
speculative : update default params (#11954)
* speculative : update default params
* speculative : do not discard the last drafted token
|
2025-02-19 13:29:42 +02:00 |
|
 Maxim Evtush
|
7b891bdc86
|
fix: typos in documentation files (#11791)
* Update ggml.c
* Update arg.cpp
* Update speculative.h
|
2025-02-10 23:21:31 +01:00 |
|
|
Georgi Gerganov
|
d9d54e498d
|
speculative : refactor and add a simpler example (#10362)
* speculative : refactor and add a simpler example
ggml-ci
* speculative : clean-up and add comments and TODOs [no ci]
* speculative : manage context in common_speculative
ggml-ci
* speculative : simplify
ggml-ci
* speculative : simplify (cont)
ggml-ci
* speculative : add --draft-min CLI arg
* speculative : minor fixup
* make : build fixes
* speculative : do not redraft previous drafts
ggml-ci
* speculative : fix the draft sampling
ggml-ci
* speculative : fix compile warning
* common : refactor args
ggml-ci
* common : change defaults [no ci]
* common : final touches
ggml-ci
|
2024-11-25 09:58:41 +02:00 |
|