pg_tiktoken
pg_tiktoken
pg_tiktoken : tiktoken tokenizer for use with OpenAI models in postgres
Overview
| ID | Extension | Package | Version | Category | License | Language |
|---|---|---|---|---|---|---|
| 1870 | pg_tiktoken | pg_tiktoken | 0.0.1 | RAG | Apache-2.0 | Rust |
| Attribute | Has Binary | Has Library | Need Load | Has DDL | Relocatable | Trusted |
|---|---|---|---|---|---|---|
--s-d-- | No | Yes | No | Yes | no | no |
| Relationships | |
|---|---|
| See Also | vectorize pg_summarize pg4ml pgml vector vchord vectorscale pg_graphql |
Packages
| Type | Repo | Version | PG Major Compatibility | Package Pattern | Dependencies |
|---|---|---|---|---|---|
| EXT | PIGSTY | 0.0.1 | 18 17 16 15 14 13 | pg_tiktoken | - |
| RPM | PIGSTY | 0.0.1 | 18 17 16 15 14 13 | pg_tiktoken_$v | - |
| DEB | PIGSTY | 0.0.1 | 18 17 16 15 14 13 | postgresql-$v-pg-tiktoken | - |
| Linux / PG | PG18 | PG17 | PG16 | PG15 | PG14 | PG13 |
|---|---|---|---|---|---|---|
el8.x86_64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
el8.aarch64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
el9.x86_64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
el9.aarch64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
el10.x86_64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
el10.aarch64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
d12.x86_64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
d12.aarch64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
d13.x86_64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
d13.aarch64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
u22.x86_64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
u22.aarch64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
u24.x86_64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
u24.aarch64 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 | PIGSTY 0.0.1 |
Source
pig build pkg pg_tiktoken; # build rpm / deb with pigInstall
Make sure PGDG and PIGSTY repo available:
pig repo add pgdg pigsty -u # add both repo and update cacheInstall this extension with pig:
pig install pg_tiktoken; # install via package name, for the active PG version
pig install pg_tiktoken -v 18; # install for PG 18
pig install pg_tiktoken -v 17; # install for PG 17
pig install pg_tiktoken -v 16; # install for PG 16
pig install pg_tiktoken -v 15; # install for PG 15
pig install pg_tiktoken -v 14; # install for PG 14
pig install pg_tiktoken -v 13; # install for PG 13Create this extension with:
CREATE EXTENSION pg_tiktoken;Last updated on