Review

The Tabnine Code Review Agent (in development)

The Code Review Agent is currently in Private Preview and available to to existing enterprise customers. Please contact our Support team to request early access.

Tabnine’s Code Review Agent enables organizations to produce higher quality, more secure code by leveraging and enforcing any given team’s unique best practices and standards for software development.

The agent checks the code against rules that align with your unique standards.

If the code doesn’t conform with your rules, the agent flags the deviations and provides guidance and suggested edits to fix the issues.

How to Use the Code Review Agent

To use the Code Review Agent, you first need to feed your coding standards and best practices into Tabnine. You can provide us one of the following:

• Best practices document in plain language that Tabnine will convert into rules for validating code.

• Access to your code repositories that serve as the gold standard for creating high-quality code. Tabnine will then extract the PR comments and other code patterns and convert them into rules for validating code.

In addition to each engineering team being able to define its own unique rules, Tabnine also offers a vast array of predefined rules any team can activate, including commonly used industry standards and language or framework-specific best practices.

Tabnine has crafted a set of predefined rules by leveraging various best practices documents, as well as thorough vetting by language-specific experts.

We have more than 140 rules for each supported language (half are related to security, while the rest verify the code’s correctness, readability, and performance).

Although some rules are generic (e.g., relevant to all Python code), most are library-specific (e.g., Cryptography, SSL, lxml), which ensures that the Code Review Agent validates not only the high-level structure of the code, but also specific functions.

Rule Characteristics

• Rules are in plain language, making it easy to review and maintain them.

• Rules are at an organizational level (i.e., the same set of rules is available to every Tabnine user within the company).

• Rules can be enabled or disabled and their severity can be set (during Private Preview, this will be done by the Tabnine team).

Guidelines for Effective Rules

A rule should be descriptive, clear, and concise, and should focus on a single pattern of code that should be followed or avoided. Here are examples of effective and ineffective rules:

• Effective rule: “Use Parameterized SQL queries: ensure that SQL queries are parameterized to prevent SQL injection attacks” provides a clear description of the rule and its purpose, which will be useful both for the AI and the developer that sees it as a comment in the PR.

• Ineffective rule: “Avoid risky concurrency code patterns that should not be used” is too general, making it more difficult for the AI agent to understand the patterns that should be enforced or avoided.

Input needed for creating rules

Each rule has the following fields. The customer needs to provide the information in the ‘description’ field, and the rest of the fields are optional (i.e. if you don’t provide the information, Tabnine will generate the information for you. You can edit the information provided by Tabnine if needed.)

1. Description: Write a short description of the desired behavior. For example, “always check the response status code for an HTTP request to ensure it was successful (e.g., response.status_code == 200 for a successful GET request)”. Alternatively, describe the behavior that needs to be avoided (e.g., “Do not ignore checking the response status code after making an HTTP request.”)

2. Language (optional): The code language(s) on which the rule should be applied

3. Severity (optional): How severe you find a violation of this rule. Options are INFO/WARNING/ERROR/CRITICAL.

4. Category (optional): What category the rule is related to: performance, security, or architecture. This will be shown in the review comment.

5. Good example (optional*): An example of a piece of code that follows the rule (e.g., a function that sends an http request and checks the status code).

6. Bad example (optional*): An example of a piece of code that violates the rule (e.g., a function that sends an http request and doesn’t check the status code).

7. Library (optional): If the rule is relevant for a specific library, or a list of libraries.

* The good and bad examples are optional, but for rules that are related to internal enterprise code (e.g., “Use a db api wrapper created in the enterprise codebase instead of the db generic library”), we highly recommend providing such examples to improve the effectiveness of the AI agent.

Examples of Rules

• Predefined Rules

  • “Never send sensitive information (such as usernames or passwords) in the URLs and query parameters as they are not secure. The recommendation is to use headers.”

  • “Only use SHA256 to securely hash data.”

  • “Use timeouts when sending HTTP requests to avoid blocks in the application.”

• Customer-Specific Rules

  • “Only use library acme_secure_api_access for accessing external APIs, do not use standard http libraries,” or “Never encrypt data yourself, always use acme_encryption_service.” (These rules use a company-specific service instead of allowing new code to be added that repeats an existing function.)

  • “Never access the users database table directly, use acme_user_service APIs.” (This rule applies another company-specific policy for service reuse.)

  • “Use acme_dropdown components instead of the react standard dropdown component.”

How the Agent Reviews Code

The Code Review Agent reviews the code in the pull request.

When developers create a pull request, the Code Review Agent checks the code in the PR against the set of rules.

If the code doesn’t conform to your expectations in any way, the agent flags it to the code reviewer and provides guidance and suggested edits to fix the issue.

Supported platforms and languages

During Private Preview, we support the following:

• Languages: Python, Java, JavaScript, TypeScript, C#, Kotlin, PHP, C++.

• Platforms: GitHub and CLI

• LLM: The underlying LLM for the Code Review Agent is GPT-4o mini