Superminaren/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-11 04:56:56 +02:00
Code Issues Packages Projects Releases Wiki Activity

llama : fix Roberta embeddings (#10856)

Browse Source 
* fix: Use gpt2 tokenizer for roberta and add eos/bos tokens

Branch: RobertaTokenizer

Signed-off-by: Gabe Goodhart <[email protected]>

* fixes to position embeddings

Signed-off-by: Sukriti-Sharma4 <[email protected]>

* map roberta-bpe to gpt-2

Signed-off-by: Sukriti-Sharma4 <[email protected]>

* fix linting

Signed-off-by: Sukriti-Sharma4 <[email protected]>

---------

Signed-off-by: Gabe Goodhart <[email protected]>
Signed-off-by: Sukriti-Sharma4 <[email protected]>
Co-authored-by: Gabe Goodhart <[email protected]>
This commit is contained in:
Sukriti Sharma
2024-12-19 15:04:51 +02:00
committed by GitHub
co-authored by Gabe Goodhart
parent 7585edbdeb
commit 2fffc52b50
2 changed files with 48 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/llama.cpp
+2 -1
View File
@@ -6592,7 +6592,8 @@ static void llm_load_vocab(
tokenizer_pre == "jina-v1-en" ||
tokenizer_pre == "jina-v2-es" ||
tokenizer_pre == "jina-v2-de" ||
tokenizer_pre == "jina-v2-code") {
tokenizer_pre == "jina-v2-code" ||
tokenizer_pre == "roberta-bpe") {
vocab.type_pre = LLAMA_VOCAB_PRE_TYPE_GPT2;
} else if (
tokenizer_pre == "refact") {