> For the complete documentation index, see [llms.txt](https://docs.tabnine.com/main/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.tabnine.com/main/administering-tabnine/managing-your-team/team-management/llm-priority-scheduling.md). # LLM Priority Scheduling {% hint style="info" %} **Availability**: Self-hosted deployments with at least one self-managed (OpenAI-compatible) model configured. **Required role**: Admin or Manager. {% endhint %} LLM Priority Scheduling lets admins assign a numeric priority (1–100) to each team, controlling which teams are served first when the self-hosted model is under heavy load. Lower numbers mean higher priority. When the model's request queue is full, requests from higher-priority teams are served before lower-priority ones. Priority only affects queued requests — it cannot interrupt a generation that is already in progress. ### **How** LLM Priority Scheduling **Works** Tabnine passes each request's priority value to the vLLM engine via its built-in priority scheduling policy. A team with priority 1 is served first. A team with priority 50, the default, receives standard handling. A team with priority 90 is most likely to see a "model busy" message during peak load. Teams with the same priority value are handled in arrival order (FIFO). Priority applies to both chat and agent requests routed through a priority-aware self-managed model. In-flight agentic flows are never interrupted; priority enforcement only applies at the next user prompt boundary. If a user belongs to multiple teams, the priority of their active team is used. LLM Priority Scheduling is also designed to support automation workflows. In addition to interactive configuration, the capability is exposed in an API-first way so organizations can manage team priorities through their own provisioning, operations, or incident-response tooling. This is especially useful in self-hosted environments where priority levels may need to change quickly during peak load or scheduled events. ### **Enabling Priority Scheduling** In the admin console, go to **Settings → AI Features**. Locate the Priority Scheduling toggle.\ This toggle is only visible when at least one self-managed model is configured. Turn the toggle **On**. When enabled, an LLM Priority column appears on the **Teams → Features** page. Turning the toggle off hides the column but preserves all stored priority values; turning it back on restores them. #### **Setting Team Priorities** Go to **Teams → Features (**[**v6.4.0**](https://docs.tabnine.com/main/administering-tabnine/managing-your-team/team-management/pages/risy3bTOlfBfgFRRXK8K#v6.4.0)**)**. In the LLM Priority column, enter a value from 1 (highest priority) to 100 (lowest priority) for each team. All teams default to 50. Changes are saved immediately. To set the same priority for multiple teams at once, select the teams using the checkboxes and use the bulk actions bar. #### ***Example Configuration*** | Team | Members | Review | Agent | LLM Priority | | ------------- | ------: | ------ | ----- | -----------: | | Platform | 12 | Yes | Yes | 1 | | SRE / On-Call | 6 | Yes | Yes | 10 | | Backend | 15 | Yes | Yes | 50 | | Frontend | 8 | Yes | Yes | 50 | | Interns | 3 | No | No | 90 | #### **What End Users See** End users do not see or control their priority level. When a request is rate-limited due to model capacity, the user sees the following message in the IDE: "The AI model is currently at capacity. Please try again in a moment." #### **Important Notes** Priority does not preempt running requests. If a lower-priority request is already being processed, a higher-priority request will not interrupt it — priority determines only the order of the waiting queue. In-flight agentic cycles are protected: if an agent flow is mid-cycle between turns, it is treated as highest priority and will not be blocked by incoming requests. Lower-priority flows are held only at their next user prompt boundary. Newly created teams default to priority 50. Disabling the toggle stops priority enforcement so all requests are treated equally, but stored priority values are preserved. #### **Prerequisites** At least one self-managed model (OpenAI-compatible) must be configured in the admin console. The Priority Scheduling toggle is hidden when only Tabnine-managed models are in use. SME verification required: Confirm with engineering whether the requirement that the self-managed model be served by a vLLM instance configured with `--scheduling-policy priority` is a customer-facing configuration step. If it is managed internally, remove this note before publishing. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.tabnine.com/main/administering-tabnine/managing-your-team/team-management/llm-priority-scheduling.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.