73bdcb395e
* Add '-ngl' support to finetune.cpp * Add fprintf in ggml_cuda_op_add When I tried CUDA offloading during finetuning following the readme, I got an assert here. This probably isn't an important case because inference later gives a warning saying you should use f16 or f32 instead when using lora * Add 'finetune.sh', which currently fails when using GPU "error: operator (): Finetuning on tensors with type 'f16' is not yet supported" * tweak finetune.sh * Suppress some warnings in ggml.c * Add f16 implementation to ggml_compute_forward_add_f16_f32 * Add an f16 case to ggml_add_cast_impl and llama_build_lora_finetune_graphs * finetune.sh: Edit comments * Add "add_f16_f32_f32_cuda" * Tweak an error message * finetune.sh: Add an optional LLAMA_MODEL_DIR variable * finetune.sh: Add an optional LLAMA_TRAINING_DIR variable * train : minor * tabs to spaces --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> Co-authored-by: cebtenzzre <cebtenzzre@gmail.com>
35 lines
1.1 KiB
Bash
35 lines
1.1 KiB
Bash
#!/bin/bash
|
|
cd `dirname $0`
|
|
cd ../..
|
|
|
|
EXE="./finetune"
|
|
|
|
if [[ ! $LLAMA_MODEL_DIR ]]; then LLAMA_MODEL_DIR="./models"; fi
|
|
if [[ ! $LLAMA_TRAINING_DIR ]]; then LLAMA_TRAINING_DIR="."; fi
|
|
|
|
# MODEL="$LLAMA_MODEL_DIR/openllama-3b-v2-q8_0.gguf" # This is the model the readme uses.
|
|
MODEL="$LLAMA_MODEL_DIR/openllama-3b-v2.gguf" # An f16 model. Note in this case with "-g", you get an f32-format .BIN file that isn't yet supported if you use it with "main --lora" with GPU inferencing.
|
|
|
|
while getopts "dg" opt; do
|
|
case $opt in
|
|
d)
|
|
DEBUGGER="gdb --args"
|
|
;;
|
|
g)
|
|
EXE="./build/bin/Release/finetune"
|
|
GPUARG="--gpu-layers 25"
|
|
;;
|
|
esac
|
|
done
|
|
|
|
$DEBUGGER $EXE \
|
|
--model-base $MODEL \
|
|
$GPUARG \
|
|
--checkpoint-in chk-ol3b-shakespeare-LATEST.gguf \
|
|
--checkpoint-out chk-ol3b-shakespeare-ITERATION.gguf \
|
|
--lora-out lora-ol3b-shakespeare-ITERATION.bin \
|
|
--train-data "$LLAMA_TRAINING_DIR\shakespeare.txt" \
|
|
--save-every 10 \
|
|
--threads 10 --adam-iter 30 --batch 4 --ctx 64 \
|
|
--use-checkpointing
|