baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
1098 commits 1 branch 0 tags 6 MiB
Commit graph

1098 commits

This branch
This branch
All branches
Author SHA1 Message Date
dependabot[bot] 485ad97909
Bump pydantic-settings from 2.0.2 to 2.0.3 
Bumps [pydantic-settings](https://github.com/pydantic/pydantic-settings) from 2.0.2 to 2.0.3.
- [Release notes](https://github.com/pydantic/pydantic-settings/releases)
- [Commits](https://github.com/pydantic/pydantic-settings/compare/v2.0.2...v2.0.3)

---
updated-dependencies:
- dependency-name: pydantic-settings
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-14 20:40:16 +00:00
dependabot[bot] e91969c888
Bump sse-starlette from 1.6.1 to 1.6.5 
Bumps [sse-starlette](https://github.com/sysid/sse-starlette) from 1.6.1 to 1.6.5.
- [Release notes](https://github.com/sysid/sse-starlette/releases)
- [Changelog](https://github.com/sysid/sse-starlette/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sysid/sse-starlette/compare/v1.6.1...v1.6.5)

---
updated-dependencies:
- dependency-name: sse-starlette
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-14 20:38:42 +00:00
dependabot[bot] 077f8ed23e
Bump fastapi from 0.101.0 to 0.101.1 
Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.101.0 to 0.101.1.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.101.0...0.101.1)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-14 20:36:56 +00:00
Billy Cao c471871d0b
make n_gpu_layers=-1 offload all layers 2023-08-13 11:21:28 +08:00
Billy Cao d018c7b01d
Add doc string for n_gpu_layers argument 2023-08-12 18:41:47 +08:00
Hannes Krumbiegel 17dd7fa8e0 Add py.typed 2023-08-11 09:58:48 +02:00
Pradyumna Singh Rathore d010ea70d2
Fixed Cuda Dockerfile 
Previously models produced garbage output when running on GPU with layers offloaded.

Similar to related fix on another repo: 331326a0e3
 2023-08-10 20:41:34 +05:30
MeouSker77 88184ed217 fix CJK output again 2023-08-09 22:04:35 +08:00
Andrei Betlen 66fb0345e8 Move grammar to function call argument 2023-08-08 15:08:54 -04:00
Andrei Betlen 1e844d3238 fix 2023-08-08 15:07:28 -04:00
Andrei Betlen 843b7ccd90 Merge branch 'main' into c0sogi/main 2023-08-08 14:43:02 -04:00
Andrei Betlen bf0c603c51 Merge branch 'main' into fix-on-m1 2023-08-08 14:38:35 -04:00
Andrei Betlen 36041c8bec Merge branch 'main' of github.com:abetlen/llama_cpp_python into main 2023-08-08 14:35:10 -04:00
Andrei Betlen d015bdb4f8 Add mul_mat_q option 2023-08-08 14:35:06 -04:00
Andrei dcc26f7f78
Merge pull request #573 from mzen17/spelling-error-patch 
Fix typo "lowe-level API" to "low-level API" in the README
 2023-08-08 14:32:34 -04:00
Andrei Betlen f6a7850e1a Update llama.cpp 2023-08-08 14:30:58 -04:00
Andrei 1dd4774ca9
Merge pull request #583 from abetlen/dependabot/pip/mkdocs-1.5.2 
Bump mkdocs from 1.5.1 to 1.5.2
 2023-08-08 14:24:40 -04:00
Andrei 03e575f6a5
Merge pull request #584 from abetlen/dependabot/pip/fastapi-0.101.0 
Bump fastapi from 0.100.1 to 0.101.0
 2023-08-08 14:24:31 -04:00
dependabot[bot] 83f8438c4f
Bump fastapi from 0.100.1 to 0.101.0 
Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.100.1 to 0.101.0.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.100.1...0.101.0)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-07 20:10:12 +00:00
dependabot[bot] 4cf2fc7d3d
Bump mkdocs from 1.5.1 to 1.5.2 
Bumps [mkdocs](https://github.com/mkdocs/mkdocs) from 1.5.1 to 1.5.2.
- [Release notes](https://github.com/mkdocs/mkdocs/releases)
- [Commits](https://github.com/mkdocs/mkdocs/compare/1.5.1...1.5.2)

---
updated-dependencies:
- dependency-name: mkdocs
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-07 20:09:55 +00:00
c0sogi 0d7d2031a9 prevent memory access error by llama_grammar_free 2023-08-07 17:02:33 +09:00
c0sogi b07713cb9f reset grammar for every generation 2023-08-07 15:16:25 +09:00
c0sogi 418aa83b01 Added grammar based sampling 2023-08-07 02:21:37 +09:00
Mike Zeng 097fba25e5
Fixed spelling error "lowe-level API" to "low-level API" 2023-08-05 02:00:04 -05:00
c0sogi ac188a21f3 Added low level grammar API 2023-08-05 14:43:35 +09:00
bretello 9f499af6b0
Update llama.cpp 2023-08-03 18:25:28 +02:00
bretello 39978ccaf5
add mul_mat_q parameter 
This also fixes a crash when loading the 70b llama2 model on MacOS with
metal and `n_gpu_layers=1`
 2023-08-03 18:24:50 +02:00
Ihsan Soydemir a5bc57e279
Update README.md 2023-08-03 16:49:45 +02:00
Ihsan Soydemir d4844b93ae
Merge pull request #2 from Isydmr/isydmr/docker-improvements 
Docker improvements
 2023-08-03 16:42:27 +02:00
Ihsan Soydemir cdab73536b
Docker improvements 2023-08-03 16:36:50 +02:00
Huge 60e85cbe46
Fix dev setup in README.md so that everyone can run it 2023-08-02 12:27:08 +02:00
Andrei 91bf8fac1b
Merge pull request #550 from abetlen/dependabot/pip/uvicorn-0.23.2 
Bump uvicorn from 0.23.1 to 0.23.2
 2023-08-01 17:39:25 -04:00
dependabot[bot] 0cc8d8282a
Bump uvicorn from 0.23.1 to 0.23.2 
Bumps [uvicorn](https://github.com/encode/uvicorn) from 0.23.1 to 0.23.2.
- [Release notes](https://github.com/encode/uvicorn/releases)
- [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/uvicorn/compare/0.23.1...0.23.2)

---
updated-dependencies:
- dependency-name: uvicorn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-01 21:25:18 +00:00
Andrei 5011ef4fc6
Merge pull request #549 from abetlen/dependabot/pip/mkdocs-material-9.1.21 
Bump mkdocs-material from 9.1.19 to 9.1.21
 2023-08-01 17:24:04 -04:00
Andrei 7fb4f0bda2
Merge pull request #548 from abetlen/dependabot/pip/fastapi-0.100.1 
Bump fastapi from 0.100.0 to 0.100.1
 2023-08-01 17:23:45 -04:00
dependabot[bot] ecb72cc0a2
Bump mkdocs-material from 9.1.19 to 9.1.21 
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.19 to 9.1.21.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.19...9.1.21)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-31 20:32:04 +00:00
dependabot[bot] 4a2f41a803
Bump fastapi from 0.100.0 to 0.100.1 
Bumps [fastapi](https://github.com/tiangolo/fastapi) from 0.100.0 to 0.100.1.
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](https://github.com/tiangolo/fastapi/compare/0.100.0...0.100.1)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-31 20:31:16 +00:00
Andrei Betlen ce57920e60 Suppress llama.cpp output when loading model. 2023-07-28 14:45:18 -04:00
Andrei Betlen a9b9f0397c Format 2023-07-28 01:53:08 -04:00
Andrei Betlen abc538fcd5 fix: annoying bug where attribute exceptions were droining out file not found exceptions 2023-07-28 01:43:00 -04:00
Andrei d89bf8c072
Merge pull request #536 from Isydmr/patch-2 
Fix OpenBLAS Docker build
 2023-07-28 01:37:23 -04:00
Ihsan Soydemir 3e77eea7ec
Fix OpenBLAS Docker build 
Current build produces the following:
`RuntimeError: Failed to load shared library '/usr/local/lib/python3.11/site-packages/llama_cpp/libllama.so': /usr/local/lib/python3.11/site-packages/llama_cpp/libllama.so: undefined symbol: cblas_sgemm`
 2023-07-27 19:44:15 +02:00
Andrei ffa81218f6
Merge pull request #531 from abetlen/dependabot/pip/uvicorn-0.23.1 
Bump uvicorn from 0.22.0 to 0.23.1
 2023-07-26 23:46:57 -04:00
dependabot[bot] 583d63351a
Bump uvicorn from 0.22.0 to 0.23.1 
Bumps [uvicorn](https://github.com/encode/uvicorn) from 0.22.0 to 0.23.1.
- [Release notes](https://github.com/encode/uvicorn/releases)
- [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
- [Commits](https://github.com/encode/uvicorn/compare/0.22.0...0.23.1)

---
updated-dependencies:
- dependency-name: uvicorn
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-27 03:33:07 +00:00
Andrei 468fccbed7
Merge pull request #532 from abetlen/dependabot/pip/black-23.7.0 
Bump black from 23.3.0 to 23.7.0
 2023-07-26 23:31:48 -04:00
dependabot[bot] ecdfe4fbd3
Bump black from 23.3.0 to 23.7.0 
Bumps [black](https://github.com/psf/black) from 23.3.0 to 23.7.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/23.3.0...23.7.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-26 19:08:28 +00:00
Andrei 6e6c275c94
Merge pull request #530 from abetlen/dependabot/pip/mkdocs-material-9.1.19 
Bump mkdocs-material from 9.1.18 to 9.1.19
 2023-07-26 15:07:21 -04:00
dependabot[bot] bfbbc8db14
Bump mkdocs-material from 9.1.18 to 9.1.19 
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.18 to 9.1.19.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.18...9.1.19)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-26 17:54:39 +00:00
Andrei 7debb505ab
Merge pull request #516 from charles-dyfis-net/poetry-no-versions-in-extras 
pyproject.toml: extras list should contain only package list, not versions
 2023-07-26 13:53:48 -04:00
Andrei e665b557fd
Merge pull request #523 from shouyiwang/tensor_split 
Update tensor_split to match llama.cpp's change
 2023-07-26 13:53:02 -04:00