mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-04 10:47:38 +02:00
Aaron Teo
e6dd0e29a6
* args: overhaul mmap/mlock/dio into single arg Signed-off-by: Aaron Teo <[email protected]> * docs: update docs with llama-gen-docs Signed-off-by: Aaron Teo <[email protected]> * chore: satisfy code quality Signed-off-by: Aaron Teo <[email protected]> * args: make the `+` sign an actual modifier now Signed-off-by: Aaron Teo <[email protected]> * chore: general code clean up + comments Signed-off-by: Aaron Teo <[email protected]> * arg: fix deprecated flags support Signed-off-by: Aaron Teo <[email protected]> * arg: quick sanity check Signed-off-by: Aaron Teo <[email protected]> * bench: sync llama-bench argument parsing Signed-off-by: Aaron Teo <[email protected]> * fix: bugfix variable behaviour + llama-bench lm column size Signed-off-by: Aaron Teo <[email protected]> * arg: inverse commands should do the opposite instead of doing nothing Signed-off-by: Aaron Teo <[email protected]> * bench: fix incorrect dash Signed-off-by: Aaron Teo <[email protected]> * bench: fix missing modifiers for deprecated flags Signed-off-by: Aaron Teo <[email protected]> * llama: switch back to thread_local Signed-off-by: Aaron Teo <[email protected]> * arg: switch back to single enum Signed-off-by: Aaron Teo <[email protected]> * docs: update arg docs Signed-off-by: Aaron Teo <[email protected]> * chore: fix missing `mlock` from llama_load_mode_from_str + cleanup llama-bench Signed-off-by: Aaron Teo <[email protected]> * llama: fix mlock not activating Signed-off-by: Aaron Teo <[email protected]> * arg: add deprecation warning when old and new flags are combined Signed-off-by: Aaron Teo <[email protected]> * arg: cont add comment for todo in the future Signed-off-by: Aaron Teo <[email protected]> * docs: sync with upstream Signed-off-by: Aaron Teo <[email protected]> * docs: re-sync with upstream again Signed-off-by: Aaron Teo <[email protected]> --------- Signed-off-by: Aaron Teo <[email protected]>
llama.cpp/examples/training
This directory contains examples related to language model training using llama.cpp/GGML. So far finetuning is technically functional (for FP32 models and limited hardware setups) but the code is very much WIP. Finetuning of Stories 260K and LLaMA 3.2 1b seems to work with 24 GB of memory. For CPU training, compile llama.cpp without any additional backends such as CUDA. For CUDA training, use the maximum number of GPU layers.
Proof of concept:
export model_name=llama_3.2-1b && export quantization=f32
./build/bin/llama-finetune --file wikitext-2-raw/wiki.test.raw -ngl 999 --model models/${model_name}-${quantization}.gguf -c 512 -b 512 -ub 512
./build/bin/llama-perplexity --file wikitext-2-raw/wiki.test.raw -ngl 999 --model finetuned-model.gguf
The perplexity value of the finetuned model should be lower after training on the test set for 2 epochs.