mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-04 02:37:27 +02:00
BartowskiandGeorgi Gerganov
85c55223ca
* Batched gemm for grid IQ quants
Style updates and a bit more performance
Clean up comments
Move code around
Vectorize IQ panel decode, lower threshold for speedup
IQ panel: single-source gather layout, gate bias, vectorize interleave
Add ggml_gemm_iqp_8x8_q8_K_p4 kernel, remove gather buffer
Move IQ panel code out of repack into iqp.cpp, clean up comments
Another comment sweep
* Add myself as iqp.* codeownder
* Remove ggml_cpu_iqp_scratch_offset and ggml_cpu_iqp_src1_conv_size
* Renaming and moving
* The other half of renaming and moving
* Move macros and ggml_cpu_iqp_mul_mat_id_min_batch definition
* Update ggml/src/ggml-cpu/iqp.h
Co-authored-by: Georgi Gerganov <[email protected]>
* Add iqp_rows work buffer
* Revert "Add iqp_rows work buffer"
This reverts commit 425542991e.
* Add NUMA fallback
* Add 10 row batch tests for IQP coverage on all grid IQ types
* Swap assert for return false in support check
* Move IQP mul_mat_id test
---------
Co-authored-by: Georgi Gerganov <[email protected]>