Tabby functions as a client-server architecture. The Tabby server is deployed as a Docker container or standalone binary on a machine with GPU acceleration (supporting NVIDIA CUDA, Apple Metal, and ROCm). The server manages model weights, performs token generation via optimized inference runtimes (such as llama.cpp), and indexes local git repositories to provide contextually relevant code snippets.
Lightweight client extensions installed in Visual Studio Code, IntelliJ IDEA, PyCharm, or Neovim communicate with the self-hosted Tabby endpoint over secure HTTP or WebSockets. As the developer types, the client sends surrounding file context to the server, which generates multi-line code suggestions with low latency. Tabby's web administrative dashboard enables administrators to configure models, manage user access tokens, and monitor server inference metrics.