Superminaren/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-04 02:37:27 +02:00
Code Issues Packages Projects Releases Wiki Activity
1,357 Commits 759 Branches 7,321 Tags
alloc-assert-fix
Commit Graph
1357 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Georgi Gerganov d119c04c15 examples : fix benchmark-matmult (#1554) 
The precision for Q4_0 has degraded since #1508
b1257 		2023-09-20 10:02:39 +03:00
Cebtenzzre 8781013ef6 make : restore build-info.h dependency for several targets (#3205) b1256 2023-09-18 10:03:53 -04:00
Erik Scholz 7ddf185537 ci : switch cudatoolkit install on windows to networked (#3236) b1255 2023-09-18 02:21:47 +02:00
Johannes Gäßler ee66942d7e CUDA: fix peer access logic (#3231) b1254 2023-09-17 23:35:20 +02:00
Johannes Gäßler 111163e246 CUDA: enable peer access between devices (#2470) b1253 2023-09-17 16:37:53 +02:00
slaren 8b428c9bc8 llama.cpp : show model size and BPW on load (#3223) b1252 2023-09-17 14:33:28 +02:00
Johannes Gäßler 578d8c8f5c CUDA: fix scratch malloced on non-main device (#3220) b1251 2023-09-17 14:16:22 +02:00
IsaacDynamo b541b4f0b1 Enable BUILD_SHARED_LIBS=ON on all Windows builds (#3215) b1250 2023-09-16 19:35:25 +02:00
Vlad 5dbc2b3213 Enable build with CUDA 11.0 (make) (#3132) 
* CUDA 11.0 fixes

* Cleaner CUDA/host flags separation

Also renamed GGML_ASSUME into GGML_CUDA_ASSUME
b1249 		2023-09-16 16:55:43 +02:00
goerch b08e75baea Fixing the last deviations from sentencepiece indicated by test-tokenizer-1 (#3170) 
* Fix für #2721

* Reenable tokenizer test for LLaMa

* Add `console.cpp` dependency

* Fix dependency to `common`

* Fixing wrong fix.

* Make console usage platform specific

Work on compiler warnings.

* Adapting makefile

* Remove trailing whitespace

* Adapting the other parts of the makefile

* Fix typo.

* Fixing the last deviations from sentencepiece indicated by test-tokenizer-1

* Simplify logic

* Add missing change...

* Fix ugly compiler warning

* llama_tokenize should accept strings containing NUL now

* Adding huichen's test case
b1248 		2023-09-16 13:41:33 +02:00
Cebtenzzre e6616cf0db examples : add compiler version and target to build info (#2998) b1247 2023-09-15 16:59:49 -04:00
Cebtenzzre 3aefaab9e5 check C++ code with -Wmissing-declarations (#3184) b1246 2023-09-15 15:38:27 -04:00
Cebtenzzre 69eb67e282 fix build numbers by setting fetch-depth=0 (#3197) b1245 2023-09-15 15:18:15 -04:00
Meng ZhangandGeorgi Gerganov 4fe09dfe66 llama : add support for StarCoder model architectures (#3187) 
* add placeholder of starcoder in gguf / llama.cpp

* support convert starcoder weights to gguf

* convert MQA to MHA

* fix ffn_down name

* add LLM_ARCH_STARCODER to llama.cpp

* set head_count_kv = 1

* load starcoder weight

* add max_position_embeddings

* set n_positions to max_positioin_embeddings

* properly load all starcoder params

* fix head count kv

* fix comments

* fix vram calculation for starcoder

* store mqa directly

* add input embeddings handling

* add TBD

* working in cpu, metal buggy

* cleanup useless code

* metal : fix out-of-bounds access in soft_max kernels

* llama : make starcoder graph build more consistent with others

* refactor: cleanup comments a bit

* add other starcoder models: 3B, 7B, 15B

* support-mqa-directly

* fix: remove max_position_embeddings, use n_train_ctx

* Update llama.cpp

Co-authored-by: Georgi Gerganov <[email protected]>

* Update llama.cpp

Co-authored-by: Georgi Gerganov <[email protected]>

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <[email protected]>

* fix: switch to space from tab

---------

Co-authored-by: Georgi Gerganov <[email protected]>
 2023-09-15 22:02:13 +03:00
Cebtenzzre 80291a1d02 common : do not use GNU zero-length __VA_ARGS__ extension (#3195) 2023-09-15 21:02:01 +03:00
Georgi Gerganov c6f1491da0 metal : fix bug in soft_max kernels (out-of-bounds access) (#3194) 2023-09-15 20:17:24 +03:00
Cebtenzzre e3d87a6c36 convert : make ftype optional in simple scripts (#3185) 2023-09-15 12:29:02 -04:00
Georgi Gerganov 8c00b7a6ff sync : ggml (Metal F32 support + reduce ggml-alloc size) (#3192) 
* sync : ggml (Metal F32 support + reduce ggml-alloc size)

ggml-ci

* llama-bench : fix ggml_cpu_has_metal() duplicate function

ggml-ci
 2023-09-15 19:06:03 +03:00
Engininja2 7e50d34be6 cmake : fix building shared libs for clang (rocm) on windows (#3176) 2023-09-15 15:24:30 +03:00
Evgeny Kurnevsky 235f7c193b flake : use pkg-config instead of pkgconfig (#3188) 
pkgconfig is an alias, it got removed from nixpkgs:
https://github.com/NixOS/nixpkgs/blob/295a5e1e2bacd6e246db8b2bb35d2a9415883224/pkgs/top-level/aliases.nix#L1408
 2023-09-15 11:10:22 +03:00
Georgi Gerganov a51b687657 metal : relax conditions on fast matrix multiplication kernel (#3168) 
* metal : relax conditions on fast matrix multiplication kernel

* metal : revert the concurrnecy change because it was wrong

* llama : remove experimental stuff
 2023-09-15 11:09:24 +03:00
Andrei 76164fe2e6 cmake : fix llama.h location when built outside of root directory (#3179) 2023-09-15 11:07:40 +03:00
Ali Tariqandmoiz.hussain c2ab6fe661 ci : Cloud-V for RISC-V builds (#3160) 
* Added Cloud-V File

* Replaced Makefile with original one

---------

Co-authored-by: moiz.hussain <[email protected]>
 2023-09-15 11:06:56 +03:00
Roland 2d770505a8 llama : remove mtest (#3177) 
* Remove mtest

* remove from common/common.h and examples/main/main.cpp
 2023-09-15 10:28:45 +03:00
Cebtenzzre 98311c4277 llama : make quantize example up to 2.7x faster (#3115) 2023-09-14 21:09:53 -04:00
jneem feea179e9f flake : allow $out/include to already exist (#3175) 2023-09-14 21:54:47 +03:00
Andrei 769266a543 cmake : compile ggml-rocm with -fpic when building shared library (#3158) 2023-09-14 20:38:16 +03:00
Asbjørn Olling cf8238e7f4 flake : include llama.h in nix output (#3159) 2023-09-14 20:25:00 +03:00
Cebtenzzre 4b8560e72a make : fix clang++ detection, move some definitions to CPPFLAGS (#3155) 
* make : fix clang++ detection

* make : fix compiler definitions outside of CPPFLAGS
 2023-09-14 20:22:47 +03:00
Alon 83a53b753a CI: add FreeBSD & simplify CUDA windows (#3053) 
* add freebsd to ci

* bump actions/checkout to v3
* bump cuda 12.1.0 -> 12.2.0
* bump Jimver/cuda-toolkit version

* unify and simplify "Copy and pack Cuda runtime"
* install only necessary cuda sub packages
 2023-09-14 19:21:25 +02:00
akawrykow 5c872dbca2 falcon : use stated vocab size (#2914) 2023-09-14 20:19:42 +03:00
bandoti 990a5e226a cmake : add relocatable Llama package (#2960) 
* Keep static libs and headers with install

* Add logic to generate Config package

* Use proper build info

* Add llama as import library

* Prefix target with package name

* Add example project using CMake package

* Update README

* Update README

* Remove trailing whitespace
b1226 		2023-09-14 20:04:40 +03:00
dylanandcanardleteer 980ab41afb docker : add gpu image CI builds (#3103) 
Enables the GPU enabled container images to be built and pushed
alongside the CPU containers.

Co-authored-by: canardleteer <[email protected]>
b1225 		2023-09-14 19:47:00 +03:00
Kerfuffle e394084166 gguf-py : support identity operation in TensorNameMap (#3095) 
Make try_suffixes keyword param optional.
 2023-09-14 19:32:26 +03:00
jameswu2014 4c8643dd6e feature : support Baichuan serial models (#3009) b1223 2023-09-14 12:32:10 -04:00
Leng YueandGeorgi Gerganov 35f73049af speculative : add heuristic algorithm (#3006) 
* Add heuristic algo for speculative

* Constrain minimum n_draft to 2

* speculative : improve heuristic impl

* speculative : be more rewarding upon guessing max drafted tokens

* speculative : fix typos

---------

Co-authored-by: Georgi Gerganov <[email protected]>
b1222 		2023-09-14 19:14:44 +03:00
goerch 71ca2fad7d whisper : tokenizer fix + re-enable tokenizer test for LLaMa (#3096) 
* Fix für #2721

* Reenable tokenizer test for LLaMa

* Add `console.cpp` dependency

* Fix dependency to `common`

* Fixing wrong fix.

* Make console usage platform specific

Work on compiler warnings.

* Adapting makefile

* Remove trailing whitespace

* Adapting the other parts of the makefile

* Fix typo.
b1221 		2023-09-13 16:19:44 +03:00
Tristan Ross 1b6c650d16 cmake : add a compiler flag check for FP16 format (#3086) b1220 2023-09-13 16:08:52 +03:00
Johannes GäßlerandHenri Vasserman 0a5eebb45d CUDA: mul_mat_q RDNA2 tunings (#2910) 
* CUDA: mul_mat_q RDNA2 tunings

* Update ggml-cuda.cu

Co-authored-by: Henri Vasserman <[email protected]>

---------

Co-authored-by: Henri Vasserman <[email protected]>
b1219 		2023-09-13 11:20:24 +02:00
FK 84e723653c speculative: add --n-gpu-layers-draft option (#3063) b1218 2023-09-13 08:50:46 +02:00
Eric SommerladeandCebtenzzre b52b29ab9d arm64 support for windows (#3007) 
Co-authored-by: Cebtenzzre <[email protected]>
b1217 		2023-09-12 21:54:20 -04:00
Johannes Gäßler 4f7cd6ba9c CUDA: fix LoRAs (#3130) b1216 2023-09-13 00:15:33 +02:00
Johannes Gäßler 89e89599fd CUDA: fix mul_mat_q not used for output tensor (#3127) b1215 2023-09-11 22:58:41 +02:00
Johannes Gäßler d54a4027a6 CUDA: lower GPU latency + fix Windows performance (#3110) b1214 2023-09-11 19:55:51 +02:00
Jhen-Jie Hong 1b0d09259e cmake : support build for iOS/tvOS (#3116) 
* cmake : support build for iOS/tvOS

* ci : add iOS/tvOS build into macOS-latest-cmake

* ci : split ios/tvos jobs
b1213 		2023-09-11 19:49:06 +08:00
Johannes Gäßler 8a4ca9af56 CUDA: add device number to error messages (#3112) b1212 2023-09-11 13:00:24 +02:00
KawrakowIwan KawrakowGeorgi Gerganov
 		f31b6f4e2d metal : PP speedup (#3084) 
* Minor speed gains for all quantization types

* metal: faster kernel_scale via float4

* Various other speedups for "small" kernels

* metal: faster soft_max vial float4

* metal: faster diagonal infinity

Although, to me it looks like one should simply
fuse scale + diagnonal infinity + soft_max on the
KQtensor.

* Another faster f16 x f32 matrix multiply kernel

* Reverting the diag infinity change

It does work for PP, but somehow it fails for TG.
Need to look more into it.

* metal: add back faster diagonal infinity

This time more carefully

* metal : minor (readibility)

---------

Co-authored-by: Iwan Kawrakow <[email protected]>
Co-authored-by: Georgi Gerganov <[email protected]>
 2023-09-11 10:30:11 +03:00
Erik Scholz 6eeb4d9083 convert: remove most of the n_mult usage in convert.py (#3098) 2023-09-10 11:06:53 -04:00
kchro3andJhen-Jie Hong 21ac3a1503 metal : support for Swift (#3078) 
* Metal support for Swift

* update

* add a toggle for arm/arm64

* set minimum versions for all platforms

* update to use newLibraryWithURL

* bump version

Co-authored-by: Jhen-Jie Hong <[email protected]>

---------

Co-authored-by: Jhen-Jie Hong <[email protected]>
 2023-09-09 17:12:10 +08:00
Jhen-Jie Hong 4fd5477955 metal : support build for iOS/tvOS (#3089) 2023-09-09 11:46:04 +03:00