From bd52f3606f85be366e5531397ac1d3fc48da892c Mon Sep 17 00:00:00 2001 From: Xuan Son Nguyen Date: Sat, 1 Aug 2026 15:38:16 +0200 Subject: [PATCH] update docs --- tools/mtmd/README-dev.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/tools/mtmd/README-dev.md b/tools/mtmd/README-dev.md index 5b94e268c1..24eba3249f 100644 --- a/tools/mtmd/README-dev.md +++ b/tools/mtmd/README-dev.md @@ -28,6 +28,12 @@ A typical pipeline of the core libmtmd is as follows: - Single image or batch is encoded, via `mtmd_encode()` or `mtmd_batch_encode()` - Get the output embeddings +## Helper + +We provide a set of helper functions via `mtmd_helper` to make using libmtmd easier. The helper provides: +- Image, audio and video file decoding (for example, decode raw JPEG into RGB bitmap) +- Manage `llama_batch` and calls to `llama_decode` + ## Audio generation support Audio generation is added to mtmd in PR [#26254](https://github.com/ggml-org/llama.cpp/pull/26254) @@ -65,10 +71,8 @@ Due to wide variety of audio generation pipelines, the `mtmd_gen_audio` system i - 10% changes inside `libmtmd` and `clip.cpp` systems - The rest downstream code (CLI, server) should have no changes at all -IMPORTANT: if the model requires any changes that doesn't fit into the existing infrastructure, **open an issue** first for discussion. Any new components must be verified to respect the API design constraints stated above +IMPORTANT: If your model needs changes that don't fit the existing infrastructure, **open an issue first for discussion**. -## Helper - -We provide a set of helper functions via `mtmd_helper` to make using libmtmd easier. The helper provides: -- Image, audio and video file decoding (for example, decode raw JPEG into RGB bitmap) -- Manage `llama_batch` and calls to `llama_decode` +No-go checklist (these will get the PR rejected and require discussion before proceeding): +- Violating the API design constraints stated above +- Adding a new model-specific binary: the API and binary surface must stay model-agnostic