haraldwolff/llama.cpp
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
llama.cpp/examples/llama.android
History
Dean 7ab7b733bb
android : fix utf8 decoding error (#5935) 
* examples: fix utf8 decoding error

some models have a tokenizer that decodes an id into an incomplete utf8 sequence, need to validate and wait for next token
one example would be: https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat-GGUF/resolve/main/qwen1_5-1_8b-chat-q4_0.gguf and and an example of the token is 18137

* android : minor

---------

Co-authored-by: zhangfuwen <zhangfuwen@foxmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2024-03-10 22:03:17 +02:00
..
app android : fix utf8 decoding error (#5935) 2024-03-10 22:03:17 +02:00
gradle/wrapper android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
.gitignore android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
build.gradle.kts android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
gradle.properties android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
gradlew android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
README.md android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00
settings.gradle.kts android : introduce starter project example (#4926) 2024-01-16 15:47:34 +02:00

README.md