text-generation-inference
Last changed
Request a free trial
Contact our team to test out this image for free. Please also indicate any other images you would like to evaluate.
Tag: latest
| Has apk? | no |
| Has a shell? | yes |
| User | 65532 |
| Environment variables | CUDA_VERSION=12.8EXLLAMA_NO_FLASH_ATTN=1HF_HOME=/dataHF_HUB_ENABLE_HF_TRANSFER=1HF_KERNELS_CACHE=/kernelsLD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64NVIDIA_DRIVER_CAPABILITIES=compute,utilityNVIDIA_VISIBLE_DEVICES=allNV_CUDA_COMPAT_PACKAGE=cuda-compat-12-8PATH=/usr/share/text-generation-inference/.venv/bin:/usr/local/cuda-12.4/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binPORT=80PYTHONPATH=/usr/lib/python3.11/site-packages:/usr/share/text-generation-inference/server:/usr/share/text-generation-inference/proto:/usr/share/text-generation-inference/.venv/lib/python3.11/site-packagesPYTHON_VERSION=3.11SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt |
| Entrypoint | /tgi-entrypoint.sh |
| CMD | — |
| Volumes | — |
| Working directory | — |
| Stop signal | — |
{
"User": "65532",
"Env": [
"CUDA_VERSION=12.8",
"EXLLAMA_NO_FLASH_ATTN=1",
"HF_HOME=/data",
"HF_HUB_ENABLE_HF_TRANSFER=1",
"HF_KERNELS_CACHE=/kernels",
"LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64",
"NVIDIA_DRIVER_CAPABILITIES=compute,utility",
"NVIDIA_VISIBLE_DEVICES=all",
"NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-8",
"PATH=/usr/share/text-generation-inference/.venv/bin:/usr/local/cuda-12.4/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"PORT=80",
"PYTHONPATH=/usr/lib/python3.11/site-packages:/usr/share/text-generation-inference/server:/usr/share/text-generation-inference/proto:/usr/share/text-generation-inference/.venv/lib/python3.11/site-packages",
"PYTHON_VERSION=3.11",
"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"
],
"Entrypoint": [
"/tgi-entrypoint.sh"
],
"Labels": {
"org.opencontainers.image.title": "text-generation-inference",
"org.opencontainers.image.url": "https://images.chainguard.dev/directory/image/text-generation-inference/overview",
"org.opencontainers.image.vendor": "Chainguard",
"dev.chainguard.image.title": "text-generation-inference",
"dev.chainguard.package.main": "py3.11-text-generation-inference",
"org.opencontainers.image.authors": "Chainguard Team https://www.chainguard.dev/",
"org.opencontainers.image.created": "2025-11-13T17:33:08Z",
"org.opencontainers.image.source": "https://github.com/chainguard-images/images-private/tree/main/images/text-generation-inference"
}
}