llama.cpp/.github/workflows
Georgi Gerganov eb34620aec
Add tokenizer test + revert to C++11 (#355)
* Add test-tokenizer-0 to do a few tokenizations - feel free to expand
* Added option to convert-pth-to-ggml.py script to dump just the vocabulary
* Added ./models/ggml-vocab.bin containing just LLaMA vocab data (used for tests)
* Added utility to load vocabulary file from previous point (temporary implementation)
* Avoid using std::string_view and drop back to C++11 (hope I didn't break something)
* Rename gpt_vocab -> llama_vocab
* All CMake binaries go into ./bin/ now
2023-03-21 17:29:41 +02:00
..
build.yml Add tokenizer test + revert to C++11 (#355) 2023-03-21 17:29:41 +02:00
docker.yml Docker - Fix publish docker image in GitHub Registry (#235) 2023-03-20 18:05:20 +01:00