* metal: fix memory leak if model is freed without any GPU operations
* metal: run dummy work only if residency sets are used
* metal: wrap function in #if defined
* metal: measure system-wide wired memory in test
* metal: always build regression test
Co-authored-by: YiChen Lv <[email protected]>
---------
Co-authored-by: YiChen Lv <[email protected]>
* fix: single-flight conversations store init
* refactor: remove unused legacy-migration util
* fix: make createSystemMessage transactional
* fix: delete message branches cascading on edit/regenerate
* fix: stop stamping lastModified on conversation metadata updates
* fix: count cascaded forks in bulk delete toast, bulkify deleteAll
* refactor: drop redundant conversation list respreads
* refactor: create conversation in a single write
* fix: use table constant in toggleConversationPin
* fix: keep the system message placeholder out of the edit form
* fix: keep focus in the system message editor after opening it
* fix: focus the main chat form after submitting a system message
* fix: update timestamp of the correct conversation on stream completion
ggml_cuda_should_use_mmq() selects MMQ purely from the quantization
type. The current MMQ configurations are designed and maintained against
a minimum of 48 KiB per-block shared memory, the limit provided by
NVIDIA Pascal GPUs and later. On devices that report less, no supported
MMQ tile fits and mul_mat_q_switch_J() aborts when every tile size
exceeds the device's per-block shared memory budget.
Disable MMQ when smpbo < 48 KiB so the caller falls back to the BLAS
path instead of hitting GGML_ABORT. Some current MUSA QY1 devices
report only 28 KiB and are covered by this guard.
Reproduced on a Moore Threads MTT S70 (arch mp_21, 28 KiB shared memory
per block) with an RWKV-7 0.1B Q8_0 model:
$ llama-bench -m rwkv7-g1d-0.1b-Q8_0.gguf -p 128 -n 0
J_best=0
ggml/src/ggml-cuda/template-instances/../mmq.cuh:1521: fatal error
(core dumped)
Only prefill (batch > 1) is affected; token generation is fine. After
the fix the same device falls back to the BLAS path:
Q8_0 pp128 1470.7 t/s, tg8 55.3 t/s (was: abort)
FP16 unchanged
Q4_K_M unchanged
This matches a -DGGML_CUDA_FORCE_CUBLAS=ON build (pp128 1464.2 t/s),
which confirms the fallback path is the one being taken.
This is not MUSA-specific: any device with less than 48 KiB per-block
shared memory is affected.
Co-authored-by: KakaruHayate <[email protected]>
Adds trace logging in server-context.cpp for slot similarity checking
during prompt cache slot selection, including skip reasons and similarity
calculation details.
Assisted-by: llama.cpp:Qwen3.6-27B
* model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2)
Adds GLM-5.2 NextN/MTP as a --spec-type draft-mtp target: nextn tensor
loading via the qwen35moe/step35-style presence probe, a graph_mtp
builder (enorm/hnorm/eh_proj + dense MLA + sigmoid-gated MoE with
shared expert + shared head with fallbacks, _s scale tensors passed
for NVFP4), t_h_nextn extraction in the trunk graph, and MTP-context
KV setup: the draft head runs dense MLA, so the MTP context uses a
plain attention KV cache holding only the nextn layer(s) (same
pattern as the hybrid Qwen3.5 MTP context) while the main context
keeps the DSA cache, now filtered to trunk layers only.
Co-Authored-By: Claude Fable 5 <[email protected]>
* convert : support --mtp/--no-mtp export for GlmMoeDsaForCausalLM (GLM-5.2)
Opt GLM-5.2 into the supports_mtp_export contract (post-#25641 shape,
mirroring HYV3Model/Step35Model): --no-mtp drops the appended NextN
block (blk.78) and its nextn_predict_layers KV; --mtp keeps only the
NextN block plus shared embeddings/norm/lm_head. Default (bundled)
output is unchanged.
Co-Authored-By: Claude Fable 5 <[email protected]>
---------
Co-authored-by: Claude Fable 5 <[email protected]>
Xuan Son Nguyen
timkhronos
Niklas WenzelandYiChen Lv
Aleksander Grygier
Jonathan Clohessy
Neo ZhangandGeorgi Gerganov
Tunahanandtnhnyc
KakaruandKakaruHayate
Titaniumtown
Alessandro de Oliveira Faria (A.K.A.CABELO)
Kaben Nanlohy
Aman Gupta
Geramy Loveless
Satinder GrewalandClaude Fable 5