llama: fold the random-access prefetch into its own feature flag

LLAMA_MMAP_RANDOM_PREFETCH existed to measure the two halves of the feature
apart, and the measurement is done: on a cold cache over the same wikitext
run, MADV_RANDOM without the batched readahead takes 94.4 s against 36.7 s
for an untouched mapping, while the pair together take 34.1 s. Suppressing
the kernel's readahead only pays if we replace it, so the split let a user
select a 2.6x regression through a documented switch.

Keep the accessor, since the call site reads better than a mode comparison,
but derive it from the mode alone.
This commit is contained in:
Daniel Han
2026-08-27 10:01:47 +00:00
parent 58325573c9
commit b8bdf73bb9
2 changed files with 4 additions and 9 deletions
+1 -8
View File
@@ -457,14 +457,7 @@ llama_mmap_random_mode llama_mmap_random_mode_get() {
}
bool llama_mmap_random_prefetch_enabled() {
// on with the feature, so the batched readahead that pays for the random hints cannot be
// left off by accident. separate only so the two halves can be measured apart.
static const bool enabled = []() {
const char * env = getenv("LLAMA_MMAP_RANDOM_PREFETCH");
return env == nullptr ? true : strcmp(env, "0") != 0;
}();
return llama_mmap_random_mode_get() != LLAMA_MMAP_RANDOM_OFF && enabled;
return llama_mmap_random_mode_get() != LLAMA_MMAP_RANDOM_OFF;
}
static size_t llama_mmap_page_size() {
+3 -1
View File
@@ -87,7 +87,9 @@ enum llama_mmap_random_mode {
llama_mmap_random_mode llama_mmap_random_mode_get();
// whether batched readahead ahead of a sparse gather is enabled (LLAMA_MMAP_RANDOM_PREFETCH)
// batched readahead ahead of a sparse gather. not separately switchable: MADV_RANDOM suppresses
// the kernel's own readahead, so without this the gather takes a synchronous fault per row and
// runs 2.6x slower than leaving the mapping alone
bool llama_mmap_random_prefetch_enabled();
struct llama_mlock {