haraldwolff
/
whisper.cpp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Support multiple GPUs (split mode) on SYCL backend (llama/5806) 

* suport multiple cards: split-mode - layer|row

* rm warning

* rebase with master, support tow new OPs, close feature for -sm=row, fix for unit test

* update news

* fix merge error

* update according to review comments
pull/1943/head
Neo Zhang Jianyu 2024-03-02 19:49:30 +08:00 committed by Georgi Gerganov
parent 422a6b16fc
commit c3bfc9bfda
No known key found for this signature in database
GPG Key ID: 449E073F9DC10735
2 changed files with 1421 additions and 789 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2205
ggml-sycl.cpp
View File

File diff suppressed because it is too large Load Diff

5
ggml-sycl.h
View File

@ -24,6 +24,11 @@ GGML_API ggml_backend_buffer_type_t ggml_backend_sycl_host_buffer_type(void);
GGML_API void ggml_backend_sycl_print_sycl_devices(void);
GGML_API GGML_CALL void ggml_sycl_get_gpu_list(int *id_list, int max_len);
GGML_API GGML_CALL void ggml_sycl_get_device_description(int device, char *description, size_t description_size);
GGML_API GGML_CALL int ggml_backend_sycl_get_device_count();
GGML_API GGML_CALL ggml_backend_buffer_type_t ggml_backend_sycl_split_buffer_type(const float * tensor_split);
GGML_API GGML_CALL void ggml_backend_sycl_get_device_memory(int device, size_t *free, size_t *total);
GGML_API GGML_CALL int ggml_backend_sycl_get_device_index(int device_id);
#ifdef __cplusplus
}
#endif