baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
1591 commits 1 branch 0 tags 5.6 MiB
Commit graph

49 commits

Author SHA1 Message Date
Kevin Cao 1f3156d4f2
fix: Check for existence of clip model path (#1264) 2024-03-08 21:00:10 -05:00
Andrei Betlen dbaba3059d fix: positional arguments only for low-level api 2024-02-26 11:31:11 -05:00
Andrei Betlen 78e536dcfe fix: typo 2024-02-26 11:14:26 -05:00
Andrei Betlen 8383a9e562 fix: llava this function takes at least 4 arguments (0 given) 2024-02-26 11:03:20 -05:00
Luke Stanley 858496224e
feat: Auto detect Mixtral's slightly different format (#1214) 2024-02-23 11:27:38 -05:00
Alvaro Bartolome 251a8a2cad
feat: Add Google's Gemma formatting via chat_format="gemma" (#1210) 
* Add Google's Gemma formatting via `chat_format="gemma"`

* Replace `raise ValueError` with `logger.debug`

Co-authored-by: Andrei <abetlen@gmail.com>

---------

Co-authored-by: Andrei <abetlen@gmail.com>
 2024-02-23 04:40:52 -05:00
Andrei Betlen 07a783779a fix: Update openbuddy prompt format. Closes #1155 2024-02-13 23:57:10 -05:00
Andrei Betlen 345215a76c fix: more chatml-function-calling fixes 2024-02-13 23:02:50 -05:00
Andrei Betlen 68fb71b6a2 fix: missing generation_prompt in chatml-function-calling 2024-02-13 03:24:41 -05:00
Andrei Betlen 4b0e3320bd fix: minor formatting bugs for chatml-function-calling 2024-02-13 03:11:35 -05:00
Andrei 153a0049d9
feat: Generic chatml Function Calling (#957) 
* Add demo notebook

* Add initial chat handler

* Update OpenAI types

* Add generic chatml function calling (wip)

* Update chatml generic function calling.

* Progress on auto-tool calls

* fix streaming functions

* Remove print statements

* fix: Suppress output from llama.cpp init and grammar creation

* Add OpenAI v1 python api compatible chat completion function

* Support non-streaming multi-tool calls

* Format

* Include function_call in response.
 2024-02-12 15:56:07 -05:00
Jeffrey Fong 901827013b
feat: Integrate functionary v1.4 and v2 models + add custom tokenizer support to Llama class (#1078) 
* convert functionary-v1 chat handler to use hf autotokenizer

* add hf_tokenizer + inteegrate functionary-v1.4 prompt template

* integrate functionary v2 prompt template

* update readme

* set up parallel function calling wip

* set up parallel function calling

* Update README.md

* Update README.md

* refactor tokenizers

* include old functionary handler for backward compatibility

* add hf_tokenizer_path in server ModelSettings

* convert functionary-v1 chat handler to use hf autotokenizer

* add hf_tokenizer + inteegrate functionary-v1.4 prompt template

* integrate functionary v2 prompt template

* update readme

* set up parallel function calling wip

* resolve merge conflict

* Update README.md

* Update README.md

* refactor tokenizers

* include old functionary handler for backward compatibility

* add hf_tokenizer_path in server ModelSettings

* Cleanup PR, fix breaking changes

* Use hf_pretrained_model_name_or_path for tokenizer

* fix hf tokenizer in streaming

* update README

* refactor offset mapping

---------

Co-authored-by: Andrei <abetlen@gmail.com>
 2024-02-07 20:07:03 -05:00
Andrei Betlen 078cca0361 fix: Pass raise_exception and add_generation_prompt to jinja2 chat template 2024-01-31 08:42:21 -05:00
Andrei da003d8768
Automatically set chat format from gguf (#1110) 
* Use jinja formatter to load chat format from gguf

* Fix off-by-one error in metadata loader

* Implement chat format auto-detection
 2024-01-29 14:22:23 -05:00
Andrei Betlen 9ae5819ee4 Add chat format test. 2024-01-29 00:59:01 -05:00
Rafaelblsilva ce38dbdf07
Add mistral instruct chat format as "mistral-instruct" (#799) 
* Added mistral instruct chat format as "mistral"

* Fix stop sequence (merge issue)

* Update chat format name to `mistral-instruct`

---------

Co-authored-by: Andrei <abetlen@gmail.com>
 2024-01-29 00:34:42 -05:00
Andrei d8f6914f45
Add json schema mode (#1122) 
* Add json schema mode

* Add llava chat format support
 2024-01-27 16:52:18 -05:00
Andrei Betlen 5b982d0f8c fix: use both eos and bos tokens as stop sequences for hf-tokenizer-config chat format. 2024-01-22 08:32:48 -05:00
Andrei Betlen 7f3209b1eb feat: Add add_generation_prompt option for jinja2chatformatter. 2024-01-21 18:37:24 -05:00
Andrei Betlen be09318c26 feat: Add Jinja2ChatFormatter 2024-01-19 15:04:42 -05:00
Andrei Betlen b8fc1c7d83 feat: Add ability to load chat format from huggingface autotokenizer or tokenizer_config.json files. 2024-01-18 21:21:37 -05:00
Fedor Moiseev 907b9e9d42
Add Saiga chat format. (#1050) 2024-01-04 18:12:58 -05:00
xaviviro cf743ec5d3
Added ChatGLM chat format (#1059) 
Co-authored-by: Xavier Vinaixa Rosello <xaviviro@MacBook-Pro-de-Xavier.local>
 2024-01-04 18:12:02 -05:00
yhfgyyf 8b4db732bd
Add qwen chat format (#1005) 2023-12-13 21:43:43 -05:00
chiensen b938cccf05
Add Pygmalion chat format (#986) 2023-12-11 20:44:04 -05:00
Gardner Bickford c2d63a7148
fix: Typo in the Open Orca chat format #874 (#947) 2023-11-26 15:39:18 -05:00
Andrei Betlen 8c3aa7858b Merge branch 'main' of github.com:abetlen/llama_cpp_python into main 2023-11-24 00:15:36 -05:00
Andrei Betlen de2e2bc083 misc fix verbose printing in functionary model 2023-11-23 20:14:23 -05:00
mrfakename d68fc07b1b
Add Zephyr format (#937) 2023-11-23 01:20:08 -05:00
caiyesd 4184835078
Add chat format to support baichuan (#938) 
Signed-off-by: caiyesd <caiyesd@gmail.com>
 2023-11-23 01:19:50 -05:00
caiyesd b8f29f4bf0
Add baichuan-2 chat format (#936) 
Signed-off-by: caiyesd <caiyesd@gmail.com>
 2023-11-22 06:08:06 -05:00
Andrei Betlen 7a3f87846b Format 2023-11-21 04:02:20 -05:00
Andrei Betlen 07e47f55ba Add support for logit_bias outside of server api. Closes #827 2023-11-21 03:59:46 -05:00
mrfakename ef65fc5ff4
Add MistralLite, Intel, and OpenChat prompt formats (#927) 
* Add MistralLite format

* Update llama_chat_format.py

* Update llama_chat_format.py
 2023-11-21 00:19:25 -05:00
TK-Master b8438f70b5
Added support for min_p (#921) 
* Added support for min_p

My small contribution to this great project.

Ref: https://github.com/ggerganov/llama.cpp/pull/3841

Closes: https://github.com/abetlen/llama-cpp-python/issues/911

* Fix for negative temp (sample_softmax)
 2023-11-20 23:21:33 -05:00
Andrei Betlen b84d76a844 Fix: add default stop sequence to chatml chat format 2023-11-10 04:24:48 -05:00
Andrei Betlen 1b376c62b7 Update functionary for new OpenAI API 2023-11-10 02:51:58 -05:00
Andrei Betlen b62c449839 Bugfix: missing response_format for functionary and llava chat handlers 2023-11-09 00:55:23 -05:00
Andrei Betlen ca4cb88351 Fix destructor NoneType is not callable error 2023-11-08 11:05:45 -05:00
Andrei Betlen b30b9c338b Add JSON mode support. Closes #881 2023-11-08 00:07:16 -05:00
Andrei Betlen 64f5153c35 Add seed parameter to chat handlers 2023-11-07 23:41:29 -05:00
Damian Stewart aab74f0b2b
Multimodal Support (Llava 1.5) (#821) 
* llava v1.5 integration

* Point llama.cpp to fork

* Add llava shared library target

* Fix type

* Update llama.cpp

* Add llava api

* Revert changes to llama and llama_cpp

* Update llava example

* Add types for new gpt-4-vision-preview api

* Fix typo

* Update llama.cpp

* Update llama_types to match OpenAI v1 API

* Update ChatCompletionFunction type

* Reorder request parameters

* More API type fixes

* Even More Type Updates

* Add parameter for custom chat_handler to Llama class

* Fix circular import

* Convert to absolute imports

* Fix

* Fix pydantic Jsontype bug

* Accept list of prompt tokens in create_completion

* Add llava1.5 chat handler

* Add Multimodal notebook

* Clean up examples

* Add server docs

---------

Co-authored-by: Andrei Betlen <abetlen@gmail.com>
 2023-11-07 22:48:51 -05:00
Andrei Betlen bbffdaebaa Refactor autotokenizer format to reusable function 2023-11-06 09:07:27 -05:00
Joe 4ff8def4d0
#717: Add support for Huggingface Autotokenizer (#790) 
Co-authored-by: Andrei <abetlen@gmail.com>
 2023-11-05 18:06:36 -05:00
earonesty 3580e2c5df
Update llama_chat_format.py (#869) 
* Update llama_chat_format.py

properly formal llama2 with first-message prompt embedded

* Update llama_chat_format.py
 2023-11-05 17:00:13 -05:00
Andrei 3af7b21ff1
Add functionary support (#784) 
* Add common grammars and json-schema-to-grammar utility function from llama.cpp

* Pass functions to format function

* Add basic functionary formatting

* Add LlamaChatHandler for more complex chat use cases

* Add function calling example notebook

* Add support for regular chat completions alongside function calling
 2023-11-03 02:12:14 -04:00
Ma, Guokai a1ac199980
Fix repeat greeting (#808) 
* fix repeated greeting

* remove seperator between role and message
 2023-10-15 13:52:21 -04:00
Andrei Betlen 305482bd41 Add chatml chat format 2023-09-30 21:01:34 -04:00
Andrei 3bca7708fb
Configurable Chat Formats (#711) 
* Add configurable default chat completion format.

* Remove chat_template file to avoid circular import

* Update llama_types

* Add chat format
 2023-09-29 19:52:04 -04:00