update docs

This commit is contained in:
Xuan Son Nguyen
2026-08-01 15:38:16 +02:00
parent 8f94fc5267
commit bd52f3606f
+10 -6
View File
@@ -28,6 +28,12 @@ A typical pipeline of the core libmtmd is as follows:
- Single image or batch is encoded, via `mtmd_encode()` or `mtmd_batch_encode()`
- Get the output embeddings
## Helper
We provide a set of helper functions via `mtmd_helper` to make using libmtmd easier. The helper provides:
- Image, audio and video file decoding (for example, decode raw JPEG into RGB bitmap)
- Manage `llama_batch` and calls to `llama_decode`
## Audio generation support
Audio generation is added to mtmd in PR [#26254](https://github.com/ggml-org/llama.cpp/pull/26254)
@@ -65,10 +71,8 @@ Due to wide variety of audio generation pipelines, the `mtmd_gen_audio` system i
- 10% changes inside `libmtmd` and `clip.cpp` systems
- The rest downstream code (CLI, server) should have no changes at all
IMPORTANT: if the model requires any changes that doesn't fit into the existing infrastructure, **open an issue** first for discussion. Any new components must be verified to respect the API design constraints stated above
IMPORTANT: If your model needs changes that don't fit the existing infrastructure, **open an issue first for discussion**.
## Helper
We provide a set of helper functions via `mtmd_helper` to make using libmtmd easier. The helper provides:
- Image, audio and video file decoding (for example, decode raw JPEG into RGB bitmap)
- Manage `llama_batch` and calls to `llama_decode`
No-go checklist (these will get the PR rejected and require discussion before proceeding):
- Violating the API design constraints stated above
- Adding a new model-specific binary: the API and binary surface must stay model-agnostic