i. Before merging PR B, make sure the branch on repo B is **not lagging behind** the main branch. This ensures that the submodule includes all the latest changes made by others.
ii. Make sure PR B is merged **before** PR A. This ensures that other developers will pick up the changes on B when making their own changes.
iii. After merging PR A, check that submodule B is still up-to-date with respect to its main branch, especially if PR B was merged more than an hour ago. Good to know: * Merging in B can be done with a merge commit or by squashing the commits. * If squashing commits in B, you must know that the original commit referenced by A becomes orphaned when the branch is deleted but remains cached by git for a while. This is usually sufficient to not require A to point to the newly-squashed commit. *If this turns out to be problematic in practice, we may have to disallow commit squashing in the future.* # Contribute rules to the Semgrep Registry Source: https://docs.semgrep.dev/contributing/contributing-to-semgrep-rules-repository Publish rules to the Semgrep Registry to share them with the Semgrep community and contribute to the field of software security. There are two ways in which you can contribute rules to the Semgrep Registry: **For users of Semgrep AppSec Platform** Contribute new rules to the Semgrep Registry through Semgrep AppSec Platform. This workflow is recommended. See [Contribute through Semgrep AppSec Platform (recommended)](#contribute-through-semgrep-appsec-platform-recommended). This workflow creates the necessary pull request for you and streamlines the whole process. **For contributors to the repository through GitHub** Contribute rules to the Semgrep Registry, or suggest changes to existing rules, through a pull request to `semgrep-rules`. See the [Contribute through GitHub](#contribute-through-github) section for detailed information. ## Contribute through Semgrep AppSec Platform (recommended) This is the recommended path for adding a new rule. To suggest a change to an existing rule, see [Update existing rules in Semgrep Registry](#update-existing-rules-in-semgrep-registry).
- `references`
- `category`
- `technology`
Additional keys required when `category` is `security`:
- `cwe`
- `owasp`
- `confidence`
- `subcategory`
- `likelihood`
- `impact`
- `vulnerability_class`
`cwe:`
`- "CWE-94: (...)"`
`category: security`
`technology:`
`- unicode`
`references`:
- [https://trojansource.codes/](https://trojansource.codes/) | | `technology` | Nested under the `metadata` field. Additional information about the technology. This helps to specify rulesets in Semgrep Registry. |
- `django`
- `docker`
- `express`
- `kubernetes`
- `nginx`
- `react`
- `terraform`
- `--no-technology--`
`technology:`
`- react` | | `category` | Nested under the `metadata` field. If you use category `security`, include additional metadata. See Including fields required by security category. |
- `best-practice`
- `correctness`
- `maintainability`
- `performance`
- `portability`
- `security`
- [OWASP DOM based XSS Prevention Cheat Sheet][OWASP-DOM-based-XSS-prevention] |
security, include additional metadata. See Including fields required by security category.
* Cross-file (interfile) analysis requires `interfile: true` under the `options` key in YAML rules. For more information, see [Creating rules that analyze across files](/semgrep-code/semgrep-pro-engine-intro/#write-rules-that-analyze-across-files-and-functions).
| Required metadata field | Values | Example use |
|---|---|---|
cwe |
A Comment Weakness Enumeration (CWE) | cwe: "CWE-502: Deserialization of Untrusted Data" |
owasp |
An OWASP Top 10 category |
owasp: |
confidence |
HIGH, MEDIUM, LOW |
confidence: MEDIUM |
likelihood |
HIGH, MEDIUM, LOW |
likelihood: MEDIUM |
impact |
HIGH, MEDIUM, LOW |
impact: HIGH |
subcategory |
vuln, audit, secure default |
subcategory: |
vulnerability\_class |
See [Vulnerability class](#vulnerability-class) for a list of sample values. Accepts custom values. |
vulnerability\_class: |
βIf a developer has to convince their manager to spend a few million dollars on advanced security tools each time they change jobs, the future is bleak.β β see our [introductory blog post](https://semgrep.dev/blog/2020/introducing-semgrep-and-r2c/) for more. Itβs important to us (and the community) that Semgrep, Inc. is able to develop a sustainable business around Semgrep to support its development, but we strongly believe the tooling itself must always be free. 2. **Open-source software**
Semgrep is [LGPL](https://tldrlegal.com/license/gnu-lesser-general-public-license-v2.1-\(lgpl-2.1\)) and powered not just by [Semgrep, Inc.](https://semgrep.dev/) but also by community of brilliant external contributors. We welcome feedback and contributions and strive to be a welcoming community for new developers. 3. **Fast**
High sloc/sec scanning speed and low startup cost. Weβll never be as fast as ripgrep but we want to get as close as we can. 4. **Code never leaves your machine**
Semgrep by default runs entirely locally (unless you set it up yourself in a server/client mode). Code never leaves your machine to be analyzed. 5. **Support every programming language**
βIf grep supports it, we will too!β This even includes those that arenβt thought of as programming languages, like Bash or Docker. 6. **Run anywhere**
Semgrep is small (\<100 MB), has minimal runtime dependencies, and should be easily installable via your programming language or operating system package manager. 7. **Keep easy things easy, and hard things possible.**
Using Semgrep to scan your code, and writing rules with which to scan, should be easy. Semgrep also smooths the process with delightful defaults and support every step of the way. But itβs also adaptable, and we welcome you using Semgrep in your own custom way. Hey, there are even [examples of scanning cat pictures out there](https://youtu.be/ybWB2Vf2V50?t=1182). 8. **Beginner-friendly**
You shouldnβt need a PhD in program analysis, or even to understand what an AST is, to be effective with Semgrep. A novice programmer should be able to write their first Semgrep rule in 60 seconds. 9. **Human-readable rules**
Rules should look like code and be easy to read and reason aboutβhopefully easier than if they were written in grep or a native linter. 10. **Self-contained rule files**
You shouldnβt need an additional plugin, dependency, or internet access to run a YAML rule. It should just work. 11. **Deterministic (implies reproducible, idempotent)**
Given the same input, Semgrep gives the same output. 12. **Runs offline**
Semgrep can run without internet access so developers can write code from airplanes or beaches. 13. **Rules are safe to run no matter where they came from**
Rules shouldnβt have the capability to run arbitrary code on your system, only to act as a function that produces a deterministic output message. 14. **Single-file analysis**
To stay fast and limit complexity, Semgrep CE draws a line at crossing file boundaries during analysis. It loses the ability to detect certain complex cross-function (interprocedural) issues, but thatβs an explicit tradeoff it makes.
Semgrep CE's goal is to catch what a senior engineer would catch in code review: Semgrep isnβt designed to find a crazy issue thatβs 300 calls from start to finish and evaded the team for 20 years. Instead, itβs designed for enforcing best-practices and automating the code review tasks that an excellent senior engineer would be capable of. For a discussion of why expressive creativity is better than a powerful engine, [see this excellent blog post by Devdatta Akhawe](https://devd.me/log/posts/static-analysis/).
As a corollary: if you design your codebase so that code in a file is safe today, it's still safe after a colleague makes a change twenty function calls away in another file. 15. **Designed to run while code is being written**
Semgrep is optimized for running in the IDE, Git commit hooks, or CIβnot for at the tail-end of a release process. 16. **A platform for program analysis**
We will expose stable internals so that researchers and engineers can develop novel program analysis work off of APIs like Semgrepβs generic AST. # Semgrep Community Edition (CE) philosophy Source: https://docs.semgrep.dev/contributing/semgrep-philosophy-1 [Semgrep CE](https://github.com/semgrep/semgrep/) is a lightweight static analysis tool for many languages. It can find bug variants with patterns that look like source code. As you think about contributing to Semgrep CE, consider these design principles that have guided Semgrep CE development so far: 1. **Free**
βIf a developer has to convince their manager to spend a few million dollars on advanced security tools each time they change jobs, the future is bleak.β β see our [introductory blog post](https://semgrep.dev/blog/2020/introducing-semgrep-and-r2c/) for more. Itβs important to us (and the community) that Semgrep, Inc. is able to develop a sustainable business around Semgrep to support its development, but we strongly believe the tooling itself must always be free. 2. **Open-source software**
Semgrep is [LGPL](https://tldrlegal.com/license/gnu-lesser-general-public-license-v2.1-\(lgpl-2.1\)) and powered not just by [Semgrep, Inc.](https://semgrep.dev/) but also by community of brilliant external contributors. We welcome feedback and contributions and strive to be a welcoming community for new developers. 3. **Fast**
High sloc/sec scanning speed and low startup cost. Weβll never be as fast as ripgrep but we want to get as close as we can. 4. **Code never leaves your machine**
Semgrep by default runs entirely locally (unless you set it up yourself in a server/client mode). Code never leaves your machine to be analyzed. 5. **Support every programming language**
βIf grep supports it, we will too!β This even includes those that arenβt thought of as programming languages, like Bash or Docker. 6. **Run anywhere**
Semgrep is small (\<100 MB), has minimal runtime dependencies, and should be easily installable via your programming language or operating system package manager. 7. **Keep easy things easy, and hard things possible.**
Using Semgrep to scan your code, and writing rules with which to scan, should be easy. Semgrep also smooths the process with delightful defaults and support every step of the way. But itβs also adaptable, and we welcome you using Semgrep in your own custom way. Hey, there are even [examples of scanning cat pictures out there](https://youtu.be/ybWB2Vf2V50?t=1182). 8. **Beginner-friendly**
You shouldnβt need a PhD in program analysis, or even to understand what an AST is, to be effective with Semgrep. A novice programmer should be able to write their first Semgrep rule in 60 seconds. 9. **Human-readable rules**
Rules should look like code and be easy to read and reason aboutβhopefully easier than if they were written in grep or a native linter. 10. **Self-contained rule files**
You shouldnβt need an additional plugin, dependency, or internet access to run a YAML rule. It should just work. 11. **Deterministic (implies reproducible, idempotent)**
Given the same input, Semgrep gives the same output. 12. **Runs offline**
Semgrep can run without internet access so developers can write code from airplanes or beaches. 13. **Rules are safe to run no matter where they came from**
Rules shouldnβt have the capability to run arbitrary code on your system, only to act as a function that produces a deterministic output message. 14. **Single-file analysis**
To stay fast and limit complexity, Semgrep CE draws a line at crossing file boundaries during analysis. It loses the ability to detect certain complex cross-function (interprocedural) issues, but thatβs an explicit tradeoff it makes.
Semgrep CE's goal is to catch what a senior engineer would catch in code review: Semgrep isnβt designed to find a crazy issue thatβs 300 calls from start to finish and evaded the team for 20 years. Instead, itβs designed for enforcing best-practices and automating the code review tasks that an excellent senior engineer would be capable of. For a discussion of why expressive creativity is better than a powerful engine, [see this excellent blog post by Devdatta Akhawe](https://devd.me/log/posts/static-analysis/).
As a corollary: if you design your codebase so that code in a file is safe today, it's still safe after a colleague makes a change twenty function calls away in another file. 15. **Designed to run while code is being written**
Semgrep is optimized for running in the IDE, Git commit hooks, or CIβnot for at the tail-end of a release process. 16. **A platform for program analysis**
We will expose stable internals so that researchers and engineers can develop novel program analysis work off of APIs like Semgrepβs generic AST. # Troubleshooting Source: https://docs.semgrep.dev/contributing/troubleshooting ## Make errors
i. Click **Create new token** to create a `SEMGREP_APP_TOKEN`, which is used to when sending results to Semgrep AppSec Platform.
ii. Copy and paste the `SEMGREP_APP_TOKEN` and its value. Store it as an environment variable or secret in your CI provider.
iii. Optional: Click **Review CI config** to see Semgrep's default YAML configuration file for your CI provider.
iv. Click **Copy snippet** and paste it into your CI provider's configuration file (the filename is typically indicated in the page). Depending on your CI provider, you may have to create a custom configuration file or use an existing one.
v. Commit the configuration file to your repository.
vi. Return to Semgrep AppSec Platform and click **Check connection**.
i. For repositories in personal accounts: Click your **profile photo > Settings > Applications**.
ii. For repositories in org accounts: Click your **profile photo > Your organizations > NAME\_OF\_ORG > Settings > GitHub Apps**.
i. All repositories will display all current and future public and private repositories.
ii. Only select repositories will display explicitly selected repositories.
i. Reference or add the [Semgrep Docker image](https://hub.docker.com/r/semgrep/semgrep). This is the recommended method.
ii. Add `pipx install semgrep` (or `uv tool install semgrep` if you use [`uv`](https://docs.astral.sh/uv/)) into your configuration file as a step or command, depending on your CI provider's syntax. See the [Python Packaging guide](https://packaging.python.org/en/latest/guides/installing-stand-alone-command-line-tools/) for more on installing standalone Python CLI tools.
| Feature | Permission required |
|---|---|
| Run Semgrep continuously in your CI workflows |
|
| Run Semgrep continuously without changing your CI workflows | Grant read access to user-selected repositories. |
| Manage user authentication with SSO | View and edit SSO configurations. |
| Receive Slack notifications | Be a Slack workspace owner, or coordinate with the team responsible. |
| Send PRs or MRs to your SCM | Edit firewall or VPN allowlists for self-hosted repositories. |
i. Under **Enter GitHub information**, indicate that you want to install the app on your **Organization**, and select the **Organization name** where the app is installed. If you have multiple GitHub organizations that you'd like to use with Semgrep, ensure that you select the **Use for multiple GitHub orgs** box.
ii. Under **Select features to enable**, indicate whether you would like to grant code access to Semgrep.
iii. Review the permissions requested by Semgrep.
iv. Click **Register a Semgrep GitHub App**. Semgrep asks if you'd like to be redirected to GitHub to continue creating the app. Click **Continue** to proceed.
v. You are taken to your GHE instance and asked to name your app. You can choose whatever name you'd like, but Semgrep recommends that you name it something that indicates that this is the Semgrep GHE app.
vi. After you name your app, choose the GHE org you want to install it on.
vii. Select the org, then click **Install**.
viii. Wait for the installation to complete. When done, you are redirected to Semgrep.
ix. Verify the installation by navigating to **Settings** > **Source code managers**. Ensure that the entry for your GitHub organization shows a **Connected** badge.
x. In GHE, you should see the app listed as installed on the **GitHub Apps** page.
* You can click **Configure** to choose the repositories to which the app has access. Additionally, you can go to **App settings** to customize the permissions granted to the app.
Choose another account by clicking the **account name** and selecting an account from the drop-down box. Then, perform the following steps to set up the connection:
- Can scan your language and that the language's maturity matches your security needs. See
Supported languages. - Provides rulesets that you can use out-of-the-box. See
Semgrep Registry.
- OpenID Connect or OAuth 2
- SAML 2.0
If you use Bitbucket or Azure Repos, you can use SSO to manage the authentication of your users, then add repositories for scanning through your CI provider. | | Scanning remote repositories through CI | Semgrep fully supports many popular CI providers. See
- GitHub
- GitLab
- Bitbucket
- Slack
- Webhooks
- Microsoft Visual Studio Code
- IntelliJ Ultimate IDEA
- Emacs
Scan monorepos
Monorepos and other large codebases often take longer to scan than smaller repositories. Scan duration depends on codebase size, rule set, and CI resourcesβnot on how long deployment takes. Semgrep supports scanning a monorepo in parts by treating each subfolder as a separate project in Semgrep AppSec Platform. This can improve scan performance and keep findings organized by component. See [Scanning a monorepo in parts](/kb/semgrep-ci/scan-monorepo-in-parts) for configuration steps. # Scan local repositories and upload findings Source: https://docs.semgrep.dev/deployment/local-to-scp-scans You can send findings (scan results) from a local repository to Semgrep AppSec Platform. The local repository is a separate **project** from its remote counterpart. This is useful for testing rules and policies, or simply scanning your own work before it is merged to your organization's trunk branch. ## Prerequisites * Locally installed `semgrep`. ## Best practices You can keep your local scans private and separate from your team by creating a Semgrep organization with only a single user. This is a **personal** org, similar to a personal account in your source code manager (SCM). This separation ensures that your findings data does not affect organizational records and trends. To create an org, perform the steps in [Create additional orgs](/deployment/create-account-and-orgs#create-additional-orgs). You don't need to perform any other steps. ## Send findings from local repository scan to Semgrep AppSec Platformi. Your **Access token**. See [User personal access tokens](https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate) for token generation information. Ensure you set the Azure DevOps SCM name to `organization_name/project_name`.
ii. The name of your **Azure DevOps Project**.
i. Optional: If you don't see the repository you want to add, click **Sync projects**.
i. Leave the **Status to check** box blank, since this value is auto-populated as you provide values in subsequent steps.
ii. Select the **Enter genre/name separately** box. Provide the following values:
a. **Genre**: `security`
b. **Name**: `semgrep-cloud-platform/scan` Once you provide the **Genre** and **Name**, Azure DevOps auto-populates **Status to check**.
iii. Choose whether the status check needs to succeed or not to complete pull requests. Selecting **Required** means that a status of `succeeded` is necessary to complete pull requests. Selecting **Optional** means that a status of `failed` will not block the completion of pull requests.
Bitbucket Cloud
You must provide a Bitbucket [workspace access token](https://support.atlassian.com/bitbucket-cloud/docs/create-a-workspace-access-token/) to Semgrep, which can be created by a user with the `Product Admin` role. Once you have Semgrep Managed Scans fully configured, you can update the token provided to Semgrep to one that's more restrictive. The scopes you must assign to the token include: * `webhook (read and write)` * `repository (read and write)` * `pullrequest (read and write)` * `project (admin)` * `account (read)` Webhook permissions are required to support diff-aware scans.Bitbucket Data Center
You must provide a Bitbucket [HTTP access token](https://confluence.atlassian.com/bitbucketserver/http-access-tokens-939515499.html) to Semgrep. The user who creates the token must be assigned the **Project Admin** role for the project you connect to Semgrep. #### Create an HTTP access tokeni. Select **Bitbucket** or **Bitbucket Data Center**.
ii. Provide your **Access token**.
iii. Provide the name of your **Bitbucket workspace**.
iv. *For Bitbucket Data Center users only*: provide the **Bitbucket Data Center URL**.
v. Click **Connect**.
i. Optional: If you do not see the repository you want to add, click **Can't find your project?** and follow the troubleshooting steps provided.
i. Follow the steps in the page to create and register both a public and private Semgrep GitHub app.
i. Optional: If you do not see the repository you want to add, click **Can't find your project?** and follow the troubleshooting steps provided.
i. Optional: If you do not see the repository you want to add, click **Can't find your project?** and follow the troubleshooting steps provided.
i. Select **GitLab Cloud** or **GitLab Self-Managed**.
ii. Provide your **Access token**.
iii. Provide your **GitLab group**.
iv. *For GitLab Self-Managed users only*: provide the **GitLab URL**.
v. Click **Connect**.
i. Optional: If you do not see the repository you want to add, click **Can't find your project?** and follow the troubleshooting steps provided.
i. Once you successfully create the connection, the role for the person who owns the token can be downgraded to **Developer**.
IP addresses
If you are behind a firewall, are using a virtual private network (VPN), or have network restrictions regarding access, you might need to add the following IP addresses to the **ingress** allowlist and **egress** allowlist: ```bash theme={null} # Ingress IP addresses (from Semgrep to your infrastructure) # and egress IP addresses (from your infrastructure to Semgrep) 35.166.231.235 52.35.248.246 52.34.137.110 44.225.64.41 ``` ### CloudFront egress IP addresses You must add **CloudFront IP addresses** to your **egress** allowlist. Refer to [Locations and IP address ranges of CloudFront edge servers](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/LocationsOfEdgeServers.html) for a list of IP addresses. ## Semgrep Network Broker The [Semgrep Network Broker](/semgrep-ci/network-broker) facilitates secure access between Semgrep and your private network. Its use can replace allowlisting the IP addresses required for **ingress** traffic from Semgrep. The Network Broker, however, only facilitates requests from Semgrep to your network. It does *not* assist with requests originating from your network to Semgrep, including egress traffic from your infrastructure to Semgrep. In other words, the only address you would have to allow inbound is `wireguard.semgrep.dev` on UDP port `51820`, or your tenant's equivalent. Depending on how restrictive your network is, you might also need to modify your egress allowlist to include the IP addresses listed in [IP addresses](#ip-addresses). For setup instructions, see [Set up Semgrep Network Broker](/semgrep-ci/network-broker). ## Features that require inbound network connectivity The following Semgrep features require Semgrep to reach resources in your network: | Feature | Guide | | :------------------------- | :--------------------------------------------------------------------------------------------------------- | | On-premise SCM connections | [Connect to on-premise orgs and projects](/deployment/connect-scm#connect-to-on-premise-orgs-and-projects) | | PR and MR comments | [PR or MR comments](/category/pr-or-mr-comments) | | Semgrep Managed Scans | [Managed Scans overview](/deployment/managed-scanning/overview) | | Semgrep Multimodal | [Semgrep Multimodal getting started](/semgrep-multimodal/getting-started) | # Source code manager permissions Source: https://docs.semgrep.dev/deployment/prepare/scm-permissions Reference for source code manager permissions required by Semgrep features Use this page as the canonical reference for Source code manager (SCM) permissions. It lists what each feature requires and where to configure access.Permissions
The following tables list **SCM roles** and **token or app scopes** required for each Semgrep feature. Unless noted otherwise, **pull request (PR)** or merge request (MR) refers to a proposed set of code changes in your SCM. See **Configure** for setup guides.- **Owner**
- **Project Collection Administrator**
- **Full access** during setup/triage-webhook enablement
- After configuration:
- `Project and Team: Read & write`
- `Pull Request Threads: Read & write`
- **Owner**
- **Project Collection Administrator**
- **Full access** on the PAT during setup
- `Code: Read`
- `Code: Status`
- `Member Entitlement Management: Read`
- `Project and Team: Read & write`
- `Pull Request Threads: Read & write`
- `Code: Read`
- `Code: Write`
- **Owner**
- **Project Collection Administrator**
- `Code: Read`
- `Code: Write`
- `Project and Team: Read & write`
- `Pull Request Threads: Read & write`
- Workspace access token: `Pull requests (read and write)`
- Repository access token: `Pull requests (read and write)`
- `webhook (read and write)`
- `repository (read and write)`
- `pullrequest (read and write)`
- `project (admin)`
- `account (read)`
- `webhook (read and write)`
- `repository (read and write)`
- `pullrequest (read and write)`
- `project (admin)`
- `account (read)`
- Cloud:
- `repository:read`
- `pullrequest:read`
- `repository:write`
- `pullrequest:write`
- Data Center:
- `repository:read`
- `repository:write`
- `repository (read and write)`
- `pullrequest (read and write)`
- [Connect a source code manager](/deployment/connect-scm)
- [Create an account](/deployment/create-account-and-orgs)
- `actions (read and write)`
- `pull requests (read and write)`
- `secrets (read and write)`
- `security events (read and write)`
- `workflows (read and write)`
- Public [`semgrep-app`](https://github.com/apps/semgrep-app) permissions
- Private Semgrep GitHub App with `Contents: Read` and permissions listed in [GitHub App permissions](#github-app-permissions)
- [GitHub Managed Scans](/deployment/managed-scanning/github)
- [Semgrep Multimodal](/semgrep-multimodal/getting-started)
- `Contents: Read`
- `Contents: Write`
- `Metadata: Read`
- `Pull requests: Write`
- [Autofix](/semgrep-code/triage-remediation/autofix)
- [Grant code access](/semgrep-appsec-platform/scm-code-access)
- `openid`
- `email`
- `profile`
- `api`
- **Owner**
- **Admin**
- `api` on the group access token
- **Maintainer**
- **Owner**
- **Admin**
- `api` on group or personal access token
- `read_repository` for code access
- [GitLab Managed Scans](/deployment/managed-scanning/gitlab)
- [Semgrep Multimodal](/semgrep-multimodal/getting-started)
- `read_repository`
- `write_repository`
- **Maintainer**
- **Owner**
- **Admin**
- `api`
- `read_repository`
- `write_repository`
GitHub App permissions
Semgrep AppSec Platform requests different GitHub App permissions depending on when you grant access. ### Permissions when signing in with GitHub Semgrep AppSec Platform requests the following standard permissions set by GitHub when you first sign in. However, not all permissions are used by Semgrep AppSec Platform.Enables Semgrep AppSec Platform to read your GitHub profile data, such as your username. **Know which resources you can access**
Semgrep does not use or access any resources when first logging in. However, you can choose to share resources at a later point to add repositories into Semgrep AppSec Platform. **Act on your behalf**
Enables Semgrep AppSec Platform to perform certain tasks **only on resources that you choose to share with Semgrep AppSec Platform**. Semgrep AppSec Platform never uses this permission and never performs any actions on your behalf, even after you have installed `semgrep-app`. For more information, see [What does 'Act on your behalf' mean?](/kb/semgrep-appsec-platform/act-on-your-behalf) and [GitHub Apps acting on your behalf](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/authorizing-github-apps) in GitHub documentation.
Enables Semgrep AppSec Platform to list repository names on the project setup page. **Reading the list of organization members**
Enables Semgrep AppSec Platform to determine who can manage your Semgrep organization based on your GitHub organization's members list. **Reading and writing pull requests**
Enables Semgrep AppSec Platform to comment about findings on PRs. Read permissions allow Semgrep AppSec Platform to automatically remove findings when the PR that introduced them is closed without merging. **Reading and writing actions**
Enables Semgrep AppSec Platform to cancel stuck jobs, rerun jobs, pull logs from jobs, and perform on-demand scanning. **Reading [GitHub Checks](https://docs.github.com/en/rest/reference/checks)**
Facilitates debugging of Semgrep AppSec Platform when configured out of [GitHub Actions](https://docs.github.com/en/actions). **Reading and writing security events**
Enables integration with GitHub Advanced Security (for example, to show Semgrep results). **Reading and writing secrets**
Enables the automatic adding of the Semgrep AppSec Platform Token to your repository secrets when onboarding projects. Note: Semgrep cannot read the values of your existing or future secrets (only the names). **Reading and writing 2 files**
Enables Semgrep AppSec Platform to configure itself to run in CI by writing to `.github/workflows/semgrep.yml` and `.semgrepignore` files. **Reading and writing workflows**
Enables Semgrep AppSec Platform to configure itself to run in CI by writing to `.github/workflows/semgrep.yml`. GitHub allows writing to files within `.github/workflows/` directory only if this permission is granted along with "Writing a single file."
Lets Semgrep list their names on the project setup page. **Reading the list of organization members**
Lets Semgrep determine who can manage your Semgrep organization based on your GitHub organization's members list. **Writing (and reading) pull requests**
Lets Semgrep comment about findings on PRs. **Writing (and reading) actions**
Allows Semgrep AppSec Platform to cancel stuck jobs, rerun jobs, pull logs from jobs, and perform on-demand scanning. **Reading checks**
Facilitates debugging of Semgrep AppSec Platform when configured out of GitHub Actions. **Writing (and reading) security events**
Enables integration with GitHub Advanced Security (for example, to show Semgrep results). **Writing (and reading) secrets**
Enables automatic adding of the Semgrep AppSec Platform Token to your repository secrets when onboarding projects. Note: Semgrep cannot read the values of your existing or future secrets (only the names). **Writing (and reading) 2 files**
Lets Semgrep configure itself to run in CI by writing to `.github/workflows/semgrep.yml` and `.semgrepignore`. **Writing (and reading) workflows**
Lets Semgrep configure itself to run in CI by writing to `.github/workflows/semgrep.yml`. GitHub allows writing to files within `.github/workflows/` only if this permission is granted along with "Writing a single file." **Read source code of the repositories you select**
Allows Semgrep Multimodal to fetch source code files on-demand to construct AI prompts.
Autofix permissions
[Semgrep Code Autofix](/semgrep-code/triage-remediation/autofix) creates a branch with AI-generated changes and opens a **draft** PR or MR. Autofix requires [Semgrep Multimodal](/semgrep-multimodal/getting-started), code **read and write** access on your SCM connection, and permission to create PRs or MRs. For SCM and plan availability, see [Supported source code managers](/getting-started/scm-support). For roles, scopes, and setup links, see the **Autofix** row for each provider in [Permissions table](/deployment/prepare/scm-permissions).GitHub App permissions and API usage
Autofix uses your **private Semgrep GitHub App** with the permissions below. Use this section for security reviews, such as determining which GitHub operations require **Contents: Read and write**). See [Grant code access](/semgrep-appsec-platform/scm-code-access) to configure **Contents: Read and write** under **Repository permissions > Contents** on the private app. #### Repository permissions | GitHub App permission | Why Autofix needs it | | :----------------------- | :---------------------------------------------------------------------------------------------------------------- | | **Contents: Read** | Clone the repository using git+https (shallow, single-branch) so Semgrep can analyze the code and generate a fix. | | **Contents: Write** | Push the Autofix branch back to the repository using git+https. | | **Metadata: Read** | Read repository metadata, including the default branch, using `GET /repos/{owner}/{repo}`. | | **Pull requests: Write** | Open a **draft** PR using `POST /repos/{owner}/{repo}/pulls`. | #### How repository contents are accessed Semgrep does **not** read or write file contents through the API's `contents` endpoints (`GET` or `PUT` `/repos/{owner}/{repo}/contents/{path}`). Autofix reads and writes code only through the **Git transport layer** (clone and push), which still requires the GitHub **Contents** permissions above. # Set a primary branch Source: https://docs.semgrep.dev/deployment/primary-branch A **primary branch** is the base or target branch for pull requests and merge requests. It is usually referred to as a **default branch** or **trunk** by your source code manager (SCM). Typical names for a primary branch include `dev`, `production`, or `develop`. In many cases, Semgrep automatically detects primary branches when they first scan your project. If you have projects (repositories) with unique primary branch names, you can set them through the Semgrep web app. A primary branch enables Semgrep to filter your findings by branch and to accurately deduplicate findings. The primary branch is also used to analyze the deployment of [secure guardrails](/secure-guardrails/secure-guardrails-in-semgrep) to your developers; findings fixed before they are merged into the primary branch reduces the overall production backlog. The following video provides an introduction and walkthrough: ## Prerequisite Ensure that the project you want to set a primary branch for has completed **at least one full scan** successfully. ## Find projects without a primary branch Projects without primary branches have an orange information iconsemgrepΒ login, you are also already signed in with the Visual Studio Code Semgrep extension. Alternatively, you can log in through your command-line interface by running `semgrep login`.
* `Semgrep: Sign out`: Log out from Semgrep AppSec Platform. Alternatively, you can sign out through your command-line interface by running `semgrep logout`.
* `Semgrep: Update rules`: For logged-in users. If the rules in the [Policies](https://semgrep.dev/orgs/-/policies) or rules included through the **Semgrep βΊ Scan: Configuration** configuration option have been changed, this command loads the new configuration of your rules for your following scan.
and many other IntelliJ products |
This appears if a finding fails the CI job. Organizations typically block PRs or MRs with failed jobs. **B - Finding description**
A human-written description always appears in a PR or MR comment, describing why your code is flagged. References may also be included to help you learn more about the finding. **C - Dataflow graph**
Some Code findings have a dataflow graph, which indicates that the finding was detected through %%taint analysis|taint\_analysis%%. The dataflow graph provides the lines of code identifying sources, sinks, and traces of unsanitized data flowing through your program. You can click the links on the boxes to take you to the lines of code. **D - Resolution or remediation section**
Various options are provided to help your resolve the finding. Depending on the type of finding, resolution options may vary. **E - Ignore instructions**
Click to view instructions about how to ignore the finding by replying to the comment. ### Type of findings by resolution **Code finding**
This type of finding is typically resolved by refactoring your code. This finding typically catches bugs, security issues, or violations of best practices. **Dependency finding**
Semgrep found that you're using a vulnerable version of a dependency. It can also detect if you're using the vulnerable function or code of the dependency. **License finding**
Semgrep has found that you're using a dependency with a license that may violate the guidelines set by your organization. **Secrets finding**
Semgrep has detected a leaked secret. Rotate the secret to resolve this finding.
**Note**: [AI-powered detection](/semgrep-code/ai-powered-detection-concepts) findings remain **Open** even when Multimodal flags them as false positives. They are **not** automatically set to **Provisionally ignored**; change the status to **Ignored** if you want to dismiss them. | | **To fix** | Findings that you have decided to fix. Commonly used to indicate that these findings are tracked in Jira or assigned to developers for further work. | | **Fixed** | Fixed findings were detected in a previous scan but are no longer detected in the most recent scan of that same branch due to changes in the code. | | **Ignored** | Findings marked as ignored are present in the code but have been labeled unimportant. Ignore false positives or deprioritized issues. Mark findings as [ignored through Semgrep AppSec Platform](/semgrep-code/triage-remediation) or by adding a [nosemgrep code comment](/ignoring-files-folders-code/#reference-summary). You can also provide a reason for ignoring a finding: **False positive**, **Acceptable risk**, **No time to fix**. | | **Closed** | Vulnerabilities that are no longer detected after a scan. This can be due to changes in the underlying rule or the code. | ### Removed findings Findings can also be **removed**. Semgrep considers a finding removed if it is not found in the most recent scan of the branch where Semgrep initially detected it due to any of the following conditions: * The rule that detected the finding isn't enabled in the policy anymore. * The rule that detected the finding was updated in a way that it no longer detects the finding. * The file path where the finding appeared is no longer found. The file path was deleted, renamed, added to a `.semgrepignore` file, added to a `.gitignore` file, or added to the list of ignored paths in Semgrep AppSec Platform. * For GitHub organization accounts: the pull request or merge request where the finding was detected has been closed without merging. Your removed findings do not count toward the fix rate or the number of findings. The removed findings also do not appear in Semgrep AppSec Platform. ### Triage behavior across refs and branches * When you triage a finding as ignored, reviewing, fixing, or reopened, Semgrep always triages across other branches and [Git references](https://git-scm.com/book/en/v2/Git-Internals-Git-References) (refs). * At scan time, there's automatic triaging that occurs in specific cases, and the behavior changes depending on the type of scan: * **Full scans**: if the current branch includes a finding that was * Previously introduced in another branch ***and*** * Triaged to a specific state
**Then** the finding in the current branch is triaged to that same state. * **Diff-aware scan**: findings introduced in a diff-aware scan are **not** automatically triaged at scan time, even if there are other instances of that finding on branches that have been triaged.
This appears if a finding fails the CI job. Organizations typically block PRs or MRs with failed jobs. **B - Finding description**
A human-written description always appears in a PR or MR comment, describing why your code is flagged. References may also be included to help you learn more about the finding. **C - Dataflow graph**
Some Code findings have a dataflow graph, which indicates that the finding was detected through %%taint analysis|taint\_analysis%%. The dataflow graph provides the lines of code identifying sources, sinks, and traces of unsanitized data flowing through your program. You can click the links on the boxes to take you to the lines of code. **D - Resolution or remediation section**
Various options are provided to help you resolve the finding. Depending on the type of finding, resolution options may vary. **E - Ignore instructions**
Click to view instructions about how to ignore the finding by replying to the comment. ## Resolve findings Different types of findings require different remediations. The following sections describe how Semgrep can help you resolve a finding. ### Rule-defined fix Some Semgrep Code findings include a [Rule-defined fix](/writing-rules/rule-defined-fix): a human-written code change defined in the rule. When available, the fix appears in your pull request or merge request comment as a **Commit suggestion** you can apply directly. This is the fastest way to fix a finding inline during code review. All Semgrep-supported SCMs provide this feature when the rule includes a fix.
/fp \ | Triage a finding as **Ignored** with the triage reason **false positive**. |
| /ar \ | Triage a finding as **Ignored** with the triage reason **acceptable risk**. |
| /other \ | Triage a finding as **Ignored** without specifying the reason; the triage reason value is set to **No triage reason**. |
| /open \ | Reopen a finding that has been triaged as **Ignored**. The comment is optional. |
**Note**: [AI-powered detection](/semgrep-code/ai-powered-detection-concepts) findings remain **Open** even when Multimodal flags them as false positives. They are **not** automatically set to **Provisionally ignored**; change the status to **Ignored** if you want to dismiss them. | | **To fix** | Findings that you have decided to fix. Commonly used to indicate that these findings are tracked in Jira or assigned to developers for further work. | | **Fixed** | Fixed findings were detected in a previous scan but are no longer detected in the most recent scan of that same branch due to changes in the code. | | **Ignored** | Findings marked as ignored are present in the code but have been labeled unimportant. Ignore false positives or deprioritized issues. Mark findings as [ignored through Semgrep AppSec Platform](/semgrep-code/triage-remediation) or by adding a [nosemgrep code comment](/ignoring-files-folders-code/#reference-summary). You can also provide a reason for ignoring a finding: **False positive**, **Acceptable risk**, **No time to fix**. | | **Closed** | Vulnerabilities that are no longer detected after a scan. This can be due to changes in the underlying rule or the code. | ### Removed findings Findings can also be **removed**. Semgrep considers a finding removed if it is not found in the most recent scan of the branch where Semgrep initially detected it due to any of the following conditions: * The rule that detected the finding is no longer enabled in the policy. * The rule that detected the finding was updated in a way that it no longer detects the finding. * The file path where the finding appeared is no longer found. The file path was deleted, renamed, added to a `.semgrepignore` file, added to a `.gitignore` file, or added to the list of ignored paths in Semgrep AppSec Platform. * For GitHub organization accounts: the pull request or merge request where the finding was detected has been closed without merging. Your removed findings do not count toward the fix rate or the number of findings. The removed findings also do not appear in Semgrep AppSec Platform. ### Triage behavior across refs and branches * When you triage a finding as ignored, reviewing, fixing, or reopened, Semgrep always triages across other branches and [Git references](https://git-scm.com/book/en/v2/Git-Internals-Git-References) (refs). * At scan time, there's automatic triaging that occurs in specific cases, and the behavior changes depending on the type of scan: * **Full scans**: if the current branch includes a finding that was * Previously introduced in another branch ***and*** * Triaged to a specific state
**Then** the finding in the current branch is triaged to that same state. * **Diff-aware scan**: findings introduced in a diff-aware scan are **not** automatically triaged at scan time, even if there are other instances of that finding on branches that have been triaged.
YOUR\_TOKEN with the login token value you copied in the previous step:
```bash theme={null}
docker run -e SEMGREP_APP_TOKEN=YOUR_TOKEN --rm -v "${PWD}:/src" semgrep/semgrep semgrep ci
```
The provided `-v` option mounts the current directory into the container to be scanned. Navigate into a different project or provide a specific local directory in the command to scan a different project.
iii. For users running Docker on **Windows**:
a. Log in to your Semgrep account (running this command will launch a browser window, but you can also use the link that's returned in the CLI to proceed):
```bash theme={null}
docker run -it semgrep/semgrep semgrep login
```
b. In the **Semgrep CLI login**, click **Activate** to proceed. Return to the CLI, and copy the login token that's shown.
c. Navigate into the root of your project, and run your first scan. Be sure to substitute YOUR\_TOKEN with the login token value you copied in the previous step:
```bash theme={null}
docker run -e SEMGREP_APP_TOKEN=YOUR_TOKEN --rm -v "%cd%:/src" semgrep/semgrep semgrep ci
```
The provided `-v` option mounts the current directory into the container to be scanned. Navigate into a different project or provide a specific local directory in the command to scan a different project.
- Query console
- Semgrep Multimodal
- Semgrep Managed Scans
- Pull request comments
- Query console
- Diff-aware scans
- Sending findings to Semgrep AppSec Platform
- Default branch identification
- Generic secrets (requires Semgrep Multimodal)
- Semgrep Multimodalβ
- Semgrep Managed Scanβ
- Autofixβ
- Query console
- Generic secrets (requires Semgrep Multimodal)
- Semgrep Multimodalβ
- Semgrep Managed Scanβ
- Autofixβ
- Query console
- Generic secrets (requires Semgrep Multimodal)
- Query console
- Query console
- Diff-aware scans and triage through PR comments require Bitbucket Data Center version 8.8 or later.
- Semgrep Managed Scans\*
- Triage through MR comments\*
- Query console
- Query console
- Query console
- Query console
- Semgrep Managed Scans\*
- Query console
- Query console
- Query console
For example:
` // nosemgrep` Β Β Β
`// nosemgrep: rule-id`
`# nosemgrep` | | | For Semgrep AppSec Platform users:
- Ignore files and folders through the use of Semgrep AppSec Platform's **Project or Global ignores**
- Override the implicit ignorelist through the use of a `.semgrepignore` file.
| | | For Semgrep Community Edition (CE) users:
Ignore files and folders through a `.semgrepignore` file | Create a `.semgrepignore` file in your **repository's root directory** or your **project's working directory** and add patterns for files and folders there. Patterns follow `.gitignore` syntax with some caveats. See [Defining ignored files and folders in `.semgrepignore`](#define-ignored-files-and-folders-in-semgrepignore). | [`.semgrepignore` sample file](https://raw.githubusercontent.com/semgrep/semgrep/develop/cli/src/semgrep/templates/.semgrepignore) | ## Understand Semgrep defaults Without user customization, Semgrep refers to the following to define ignored files and folders: * Semgrep's default `.semgrepignore` file * Your repository's `.gitignore` file (if it exists) * For Semgrep AppSec Platform users: each project (repository or subfolder in monorepo) in Semgrep has a list of ignored files and folders in its project details page. In the absence of a user-generated `.semgrepignore`, Semgrep refers to [its repository's default template](https://github.com/semgrep/semgrep/blob/develop/cli/src/semgrep/templates/.semgrepignore): ```text theme={null} # Administrative folder or file used by popular version control systems .git .svn .hg _darcs CVS # Paths to files and folders that are typically large and ignorable build/ vendor/ dist/ *.min.js .env/ .tox/ # Package managers node_modules/ .npm/ .yarn/ .venv/ _opam/ _build/ _cargo/ # Note that PHP composer uses vendor/ and C++ conan uses build/ # .venv is used both by Go and Python. # Common test paths test/ tests/ testsuite/ *_test.go ``` ## Override defaults The default `.semgrepignore` file causes Semgrep to skip these folders: * `/tests`, `/test` * `/vendors` To include these folders:
Semgrep docs
Find bugs and reachable dependency vulnerabilities in code. Enforce your code standards on every commit.
Scan with Semgrep AppSec Platform
Deploy static application security testing (SAST), software composition analysis (SCA), and secrets scans from one platform.
Supported languages
| Product | Languages |
|---|---|
| Semgrep Code |
Generally available (GA)
C and C++ β’ C# β’ Generic β’ Go β’ Java β’ JavaScript β’ JSON β’ Kotlin β’ Python β’ TypeScript β’ Ruby β’ Rust β’ JSX β’ PHP β’ Scala β’ Swift β’ Terraform
Beta
APEX β’ Elixir
Experimental
Bash β’ Cairo β’ Circom β’ Clojure β’ Dart β’ Dockerfile β’ Hack β’ HTML β’ Jsonnet β’ Julia β’ Lisp β’ Lua β’ Move on Aptos β’ Move on Sui β’ OCaml β’ R β’ Scheme β’ Solidity β’ YAML β’ XML |
| Semgrep Supply Chain |
Generally available reachability
C# β’ Go β’ Java β’ JavaScript and TypeScript β’ Kotlin β’ PHP β’ Python β’ Ruby β’ Rust β’ Scala β’ Swift
Languages without support for reachability analysis
Dart β’ Elixir |
| Semgrep Secrets | Language-agnostic; can detect 630+ types of credentials or keys. |
See Supported languages documentation for more details.
What's New
See the latest Semgrep product highlights and weekly release notes.
ii. In that DefectDojo product, create an [engagement](https://defectdojo.github.io/django-DefectDojo/usage/models/#engagement), called `semgrep`. This is a CI/CD engagement type and the name designates the CI/CD tool used.
python3 integrations/defectdojo/import\_semgrep\_to\_defect\_dojo.py --host DOJO\_URL --product PRODUCT\_NAME --engagement ENGAGEMENT\_NAME --report REPORT\_FILEββi. Sign in to Semgrep AppSec Platform.
ββii. Navigate to **[Settings > Access > Login methods](https://semgrep.dev/orgs/-/settings/access/loginMethods)**. ββiii. Click **Add SSO configuration** and select **SAML2 SSO**.
ββiv. Copy the **Audience URL (SP Entity ID)** value from Semgrep AppSec Platform. Return to **Basic SAML Configuration**. Click **Add identifier** to paste this value as the **Identifier (Entity ID)**.
ββv. Copy the **SSO URL** value from Semgrep AppSec Platform. Return to **Basic SAML Configuration**. Click **Add reply URL** to paste this value as the **Reply URL (Assertion Consumer Service URL)**.
ββi. Click **Add new claim**.
ββii. Enter `name` in the **Name** field.
ββiii. For the **Source attribute** drop-down box, select `user.displayname`.
ββiv. Click **Save**.
ββi. Click **Add new claim**
ββii. Enter `email` in the **Name** field.
ββiii. From the **Source attribute** drop-down box, select `user.mail`.
ββiv. Click **Save**.
ββi. Provide the **Display name** and the **Email domain** you are using for the integration.
ββii. Copy the **Login URL** value from Microsoft Entra ID and paste it in into Semgrep AppSec Platform's **IDP SSO URL** field.
ββiii. Copy and paste the **Microsoft Entra ID Identifier** value into Semgrep AppSec Platform's **IdP Issuer ID** field.
ββiv. In Entra ID's **SAML-based Sign-on** page, click **Download** to obtain the **Certificate (Base64)**.
ββv. In Semgrep AppSec Platform, under **Upload/Paste certificate**, click **Browse** and then select the certificate you downloaded.
https\://MY\_COMPANY.semgrep.dev
## To log in to the correct tenant site
Set the environment variable `SEMGREP_APP_URL` before calling the `semgrep login` function.
```bash theme={null}
export SEMGREP_APP_URL=https://mycompany.semgrep.dev
semgrep login
```
If you frequently log in from the command line, set the SEMGREP\_APP\_URL variable in your shell initialization file, such as `~/.zshrc` or `~ /.bash_profile`, depending on your operating system.
# SAML SSO error BadRequest: Missing attribute
Source: https://docs.semgrep.dev/kb/semgrep-appsec-platform/sso-attribute-error
When setting up SAML-based SSO for Semgrep AppSec Platform, you may see the following error:
```text theme={null}
Semgrep encountered an SSO error BadRequest.
Could not process SAML parameters. Missing attribute: email
```
Semgrep AppSec Platform requires two SAML attributes to be sent: `name` and `email`. If either one is missing, this error appears during the login process. For example, if the `name` attribute is missing, the message reads `Missing attribute: name`.
When you see this error, review your SAML configuration or the SAML assertion content. Most commonly, when setting up the SAML app, you provided the right information (the user's name and email) but the name or namespace of the attributes isn't an exact match to `name` or `email`. For example, you are sending the user's full name, but the attribute is called `user.fullName`. You should rename the attribute to `name`.
Review step 4 of the [SAML setup process](/deployment/sso/#saml-20) for guidance in setting up the attributes as required by Semgrep. For Microsoft Entra ID, see steps 4 and 5 of [Set up SAML SSO with Microsoft Entra ID](/kb/semgrep-appsec-platform/saml-microsoft-entra-id).
# Semgrep in CI
Source: https://docs.semgrep.dev/kb/semgrep-ci
ββi. In the **Repository** list, select **Push**.
ββii. In the **Pull request** list, select **Created** and **Updated**.
ββi. Click **Add**.
ββii. Select one of the following: **Bitbucket Cloud Pull Request**, **Bitbucket Server Pull Request**, or **Push**.
ββiii. In **Select an Action**, select **Created**.
ββiv. Click **Add** again, and select the same trigger as before: **Bitbucket Cloud Pull Request**, **Bitbucket Server Pull Request**, or **Push**.
ββv. In **Select an Action**, select **Updated**.
ββi. In **SCM**, select **Git**.
ββii. In **Repositories > Repository URL**, enter your Bitbucket repository URL.
ββiii. In **Branch Specifier (blank for 'any')**, enter the name of your main branch.
ββiv. In **Script Path**, enter `Jenkinsfile`.
ββi. Add your Bitbucket **Repository URL**
ββii. Add the **Credentials** needed to check out your sources
ββiii. Add the **Branches to build**
ββi. Click **Add**.
ββii. Select one of the following: **Bitbucket Cloud Pull Request** or **Bitbucket Server Pull Request**.
ββiii. In **Select an Action**, select **Created**.
ββiv. Click **Add** again, and select the same trigger as before: **Bitbucket Cloud Pull Request** or **Bitbucket Server Pull Request**.
ββv. In **Select an Action**, select **Updated**.
ββvi. Click **Add > Push**.
ββi. In the **Environment** section, select **Use secret text(s) or file(s)**.
ββii. Under **Bindings**, select **Secret text**.
ββiii. Set **Variable** to `SEMGREP_APP_TOKEN`.
ββiv. Under **Credentials > Specific credentials**, choose the defined credential for the token.
ββv. Click **Add** to save your changes.
git merge-base --all SHA FETCH\_HEADgit cat-file -e REFUSERNAME).
For more information on all of these variables see GitLab documentation [Predefined variables reference](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html).
Example with sample values:
```sh theme={null}
export GITLAB_CI='true'
export CI_PROJECT_PATH="gitlab-org/gitlab-foss"
export CI_MERGE_REQUEST_PROJECT_URL="https://example.com/gitlab-org/gitlab-foss"
export CI_PROJECT_URL="$CI_MERGE_REQUEST_PROJECT_URL"
export CI_COMMIT_SHA="1ecfd275763eff1d6b4844ea3168962458c9f27a"
export CI_COMMIT_REF_NAME="main"
export CI_MERGE_REQUEST_TARGET_BRANCH_NAME="main"
export CI_JOB_URL="https://gitlab.com/gitlab-examples/ci-debug-trace/-/jobs/379424655"
export CI_PIPELINE_SOURCE='merge_request_event'
export CI_MERGE_REQUEST_IID="1"
export CI_MERGE_REQUEST_DIFF_BASE_SHA="1ecfd275763eff1d6b4844ea6874447h694gh23d"
export CI_MERGE_REQUEST_TITLE="Testing branches"
```
# Why are there new source code manager (SCM) connections that I didn't manually configure listed in Semgrep AppSec Platform?
Source: https://docs.semgrep.dev/kb/semgrep-ci/new-scm-connections
If you initiate Semgrep scans using GitHub Actions or GitLab CI/CD pipeline, Semgrep may automatically create new SCM connections and add the accompanying projects to Semgrep AppSec Platform. This can happen if the CI job has sufficient permissions through the access token you provide to create the connection between Semgrep and GitHub or GitLab.
The projects associated with the newly created SCM connections are listed in Semgrep AppSec Platform on the **Projects > Not scanning** page. They are not automatically scanned by Semgrep.
# Does Semgrep scan compressed files or other non-code files?
Source: https://docs.semgrep.dev/kb/semgrep-ci/scan-compressed-files-artifacts
Semgrep is a pre-build security tool optimized to search for code and text patterns. It does not scan the files within a compressed archive, nor does it scan binaries (built files).
## How can I scan the files inside a compressed archive file?
To scan code or text files that are stored in a compressed archive file with Semgrep, uncompress the files before performing the scan. When the scan is complete, delete the temporary files that were created.
For local scans, this can be done manually. For scans in CI, add appropriate actions to the CI config.
When implementing this method, it's optimal to place the compressed files in a consistent location, so that Semgrep can detect that any findings within the temporary files are the same across scans.
### What are the limitations of this approach?
When possible, Semgrep AppSec Platform generates hyperlinks to a finding's location within a repository and file. If the file is not persistent in the repository, and is scanned at a temporary path, then the hyperlink will lead to that temporary path and will not work properly. This may make it more difficult for developers to identify where and how to fix issues identified in the temporary files.
Currently, it is not possible to uncompress files before running a scan in [Semgrep Managed Scans](/deployment/managed-scanning/overview).
# Scanning a monorepo in parts
Source: https://docs.semgrep.dev/kb/semgrep-ci/scan-monorepo-in-parts
With default CI configurations, monorepos will be scanned as a single project in Semgrep. However, monorepos very often contain a large amount of code and the code is usually divided into to different components or modules.
As such, it can be helpful to scan a monorepo in parts for multiple reasons:
* To improve scan performance in CI and reduce CI run times
* To logically split the monorepo to simplify managing findings
semgrep scan --config='SEMGREP\_RULES\_FOLDER/'
semgrep scan --config='SEMGREP\_RULES\_FOLDER/NAME\_OF\_IMPROVED\_RULE.yaml'### Remove the rule from the scan Delete the rule from the folder containing your Semgrep rules. ## Use advanced analyses and Pro rules Optimizing rules can be a time-consuming process. Often, rules are not necessarily noisy, but lack additional analysis to detect true positives while ignoring false positives. [Semgrep Code](/semgrep-code/overview/) provides cross-function (interprocedural) and cross-file (interfile) analyses. These analyses both reduce false positives and detect true positives that Semgrep Community Edition (CE) can't find. For some languages and frameworks, such as Java or the Python Django framework, Semgrep also provides advanced analyses that take into account the language's characteristics, framework-specific dataflows, and the like. These analyses are available by default once you've signed in to Semgrep.
semgrep ci --pro
x.y.z is a placeholder for a version string.
## Running different versions using pipx
Install a specific Semgrep version using `pipx`'s version syntax:
pipx install semgrep==x.y.zpipx install --force semgrep==x.y.zuv tool install semgrep==x.y.zuvx semgrep@x.y.z --versiondocker pull semgrep/semgrep:x.y.zdocker run --rm -v "/PATH/TO/SRC:/src" semgrep/semgrep:x.y.z semgrep --config=autoYou searched for:
``` Here, the code takes the value of `q` from the URL and inserts it directly into the page. If someone navigates to `https://semgrep.dev/search?q=Β Β Β Β Β Β The [Semgrep Registry](https://semgrep.dev/explore) is a collection of rules and rulesets: * All rules, which includes both Community and Pro rules, listed in the [semgrep-rules](https://github.com/semgrep/semgrep-rules) repository are licensed under [Semgrep Rules License v.1.0](https://semgrep.dev/legal/rules-license). They are available only for internal business use. Vendors cannot use Semgrep-maintained rules in competing products or SaaS offerings. Individuals, security consultants, and companies are welcome to use the rules internally. * Rules from third-party repositories in the [Semgrep Registry](https://semgrep.dev/explore) inherit the licenses of their source repositories. These licenses are displayed within the rule definition in the editor. For example: [Rules written by Trail of Bits](https://semgrep.dev/p/trailofbits) security experts licensed under AGPL-3.0 license. **Semgrep AppSec Platform**
Β Β Β Β Β Β Proprietary. See [Terms of Service](https://semgrep.dev/terms). **Semgrep Code**
Β Β Β Β Β Β Proprietary. See [Terms of Service](https://semgrep.dev/terms). **Semgrep Secrets**
Β Β Β Β Β Β Proprietary. See [Terms of Service](https://semgrep.dev/terms). **Semgrep Supply Chain**
Β Β Β Β Β Β Proprietary. See [Terms of Service](https://semgrep.dev/terms). **Semgrep Community Edition (CE)**
Β Β Β Β Β Β The Semgrep CE engine is an open source project licensed under [LGPL 2.1](https://github.com/semgrep/semgrep/blob/develop/LICENSE). The proprietary extension of Semgrep CE is Semgrep Code, see also [Terms of Service](https://semgrep.dev/terms). ## License Semgrep for use If you are interested in using Semgrep products for your own solutions and code analysis tools, send us an email at [partnerships@semgrep.com](mailto:partnerships@semgrep.com) # Semgrep metrics Source: https://docs.semgrep.dev/metrics Semgrep CLI may collect aggregate metrics to help improve the product. This document describes: * [the principles that guide our data-collection decisions](#principles) * [how to change when Semgrep sends metrics](#automatic-collection-opt-in-and-opt-out) * [what data is not collected](#data-not-collected) * [what data is collected](#data-collected-as-metrics) # Semgrep metrics Semgrep CLI may collect aggregate metrics to help improve the product. This document describes: * [the principles that guide our data-collection decisions](#principles) * [how to change when Semgrep sends metrics](#automatic-collection-opt-in-and-opt-out) * [what data is not collected](#data-not-collected) * [what data is collected](#data-collected-as-metrics) ## Principles These principles inform our decisions around data collection: 1. **Transparency**: Collect and use data in a way that is clearly explained to the user and benefits them 2. **User control**: Put users in control of their data at all times 3. **Limited data**: Collect what is needed, pseudoanonymize where possible, and delete when no longer necessary ## Automatic collection, opt-in, and opt-out ```sh theme={null} $ semgrep --config=myrule.yaml # β no metrics (loading rules from local file) $ semgrep --config=p/python # β metrics enabled (fetching Registry) $ semgrep login && semgrep ci # β metrics enabled (logged in to semgrep.dev) $ semgrep install-semgrep-pro # β metrics enabled (requires being logged in) ``` Semgrep does **not** enable metrics when running with only local configuration files or command-line search patterns. Semgrep does enable metrics if rules are loaded from the [Semgrep Registry](https://semgrep.dev/r). This helps maintainers improve the correctness and performance of registry rules. Metrics may also be configured to be sent on every run, or never sent. To configure metrics, pass the `--metrics` option to Semgrep: * `--metrics auto`: (default) metrics are sent whenever rules are pulled from the [Semgrep Registry](https://semgrep.dev/r) or the user is logged in. * `--metrics on`: metrics are sent on every Semgrep run * `--metrics off`: metrics are never sent Alternatively, set the `SEMGREP_SEND_METRICS` environment variable to `auto`, `on`, or `off`. Note that certain Semgrep integrators turn on metrics for every run. For example, [GitLab's Semgrep SAST analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) uses `--metrics on` by default. ## Data NOT collected ### Data NOT collected ever We strive to balance our desire to collect data for improving Semgrep with our users' need for privacy and security. After all, we are a security tool! The following never leave your environment and are not sent or shared with anyone. * Source code * Private rules ### Data NOT collected unless explicitly requested The following data will never leave your environment as part of metrics. * Filenames * Git commit hashes, timestamps, messages, authors * User-identifiable data about Semgrepβs findings in your code, including finding messages This data will be sent to Semgrep AppSec Platform only if you explicitly request it, such as with `semgrep login && semgrep ci` to connect with Semgrep AppSec Platform. Even in that case, your source code and private rules will never be sent. ## Data collected as metrics Semgrep CLI collects data to improve the user experience. Five types of data are collected: ### Environmental Environmental data provide contextual data about Semgrep CLIβs runtime environment, as well as information that helps debug any issues users may be facing; e.g. * How long the command took to run * The version of Semgrep CLI * An [anonymous user ID](#anonymous-user-id) that identifies the machine * IP address that triggers a run * Value of the CI environment variable, if set * How the Semgrep CLI was installed (`pip`, `homebrew`, `docker`, or `unknown`), detected heuristically from the install path and environment variables * Pseudoanonymized hash of the scanned projectβs name * Pseudoanonymized hash of the rule definitions run * Pseduoanonymized hash of the config option. *(Note that when a config option downloads a ruleset from the [https://semgrep.dev](https://semgrep.dev) registry, [feature usage metrics](#feature-usage) still include the ruleset name in plain text.)* ### Performance Performance data enable understanding of which rules and types of files are slow in the aggregate so Semgrep, Inc can improve the program-analysis engine, query optimizer, and debug slow rules; e.g. * Runtime duration * Duration of individual phases (e.g. parsing) * Total number of rules * Total number of files * Project size in bytes ### Parse Rates Aggregated parse rate information is reported on a per-language basis; e.g., * Number of targeted files * Number of files without any parse-related error * Number of bytes across targeted files * Number of bytes without any parse-related error ### Errors High-level error and warning classes encountered when run; e.g. * Semgrepβs return code * The number of errors * Compile-time error names, e.g., MaxFileSizeExceeded, SystemOutOfMemory, UnknownFileEncoding ### Value Data that indicate how useful a run is for the end user; e.g. * Number of raised findings * Number of ignored findings * Pseudoanonymized hashes of the rule definitions that yield findings * The [features used](#feature-usage) during the scan * The engine type requested for the scan ### Extension Additional data is reported when used in conjunction with an IDE integration, such as the [Semgrep VS Code Extension](https://github.com/semgrep/semgrep-vscode), that help us understand what IDEs are used and how helpful the integrations are for users; e.g. * IDE being used * Version of IDE integration * Number of fixes triggered through the integration * Number of findings ignored through the integration Note: For all officially supported Semgrep IDE integrations, these metrics can be disabled via settings in the IDE. By default these settings follow any global telemetry/metrics settings the user may have already set for the IDE itself. ### Pro Engine installation When running `semgrep install-semgrep-pro` (which requires being logged in), additional data is reported to help us understand how many Pro Engine installations happen and how many succeed; e.g. * Whether the installation completed successfully * Which installation step failed, as a fixed identifier such as `download-403` or `version-check-failed` (never a free-form error message) ### Pseudoanonymization Certain identifying data (e.g. project URLs) are pseudoanonymized before being sent to the Semgrep, Inc backend. "Pseudoanonymized" means the data are transformed using a deterministic cryptographically secure hash. When the input data are unknown, this hash is expensive to reverse. However, when input data are known, a reverse dictionary of identifiers to hashes can be built. Hence, data are anonymous only when the source values are unknown. We use a deterministic hash to: * Track performance and value improvements over successive runs on projects * Remove test data from our metrics Using a deterministic hash, however, implies: * An entity that independently knows the value of an input datum AND who has access to Semgrep, Inc's metrics data could access metrics for that known datum Semgrep, Inc will: * Treat collected metrics data as secret, using application-security best practices, including (but not limited to) * Encryption during transit and rest * Strict access control to data-storage systems * Application-security-policy requirements for third parties (e.g. cloud-service providers; see "data sharing" below) * Only correlate hashed data to input data when these inputs are already known to Semgrep, Inc (e.g. publicly available project URLs for open-source projects, or projects that log in to the Semgrep Registry) ## Description of metrics fields | Category | Field | Description | Use Case | Example Datum | Type | | ----------- | --------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | | Environment | | | | | | | | Timestamps (started/sent) | Time when the event fired | Understanding tool usage over time | 2021-05-10T21:05:06+00:00 | String | | | Event ID | A random UUID generated when sending the event. | Deduplicating events in case of issues during transmission | 222bcccd-9dc2-4d10-ac3a-5692460e77ee | String | | | [Anonymous User ID](#anonymous-user-id) | A random UUID generated on first run. | Unique users per ruleset and feature. Understanding percentage of logged in users. | 5f52484c-3f82-4779-9353-b29bbd3193b6 | String | | | Version | Semgrep version being used | Reproduce and debug issues with specific versions | 0.51.0 | String | | | Project hash | One-way hash of the project URL | Understand performance and accuracy improvements | `c65437265631ab2566802d4d90797b27fbe0f608dceeb9451b979d1671c4bc1a` | String | | | Rules hash | One-way hash of the rule definitions | Understand performance improvements | `b03e452f389e5a86e56426c735afef13686b3e396499fc3c42561f36f6281c43` | String | | | Config hash | One-way hash of the config argument | Understand performance and accuracy improvements | `ede96c41b57de3e857090fb3c486e69ad8efae3267bac4ac5fbc19dde7161094` | String | | | Is authenticated | Whether the user logged in to semgrep.dev with `semgrep login` | Understand popularity of logged in features | `false` | Boolean | | | Deployment ID | The ID organization associated with the logged in account | Understand popularity of logged in features by organization | 1234 | Number | | | Integration name | If Semgrep is being called by another tool, optional name of that integration | Reproduce and debug issues specific integrations | `gitlab` | String | | | CI | Notes if Semgrep is running in CI and the name of the provider | Reproduce and debug issues with specific CI providers | GitLabCI v0.13.12 | String | | | Install method | How the Semgrep CLI was installed, detected heuristically | Understand which distribution channels are used; debug channel-specific issues | `pip` | String | | | Client IP | IP address that triggered a run | Understand broad ruleset usage | 0.0.0.0 | String | | | | | | | | | Performance | | | | | | | | Duration | How long the command took to run | Understanding aggregate performance improvements and regressions | 14.13 | Number | | | Total Rules | Count of rules | Understand how duration is affected by #rules | 137 | Number | | | Total Files | Count of files | Understand how duration is affected by #files | 4378 | Number | | | Total Bytes | Summation of target file size | Understand how duration is related to total size of all target files | 40838239 | Number | | | Rule Stats | Performance statistics (w/ rule hashes) for slowest rules | Debug rule performance | `[{"ruleHash": "7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166","parseTime": 0,"matchTime": 0.05480456352233887,"runTime": 0.20836973190307617,"bytesScanned": 0}]` | StatsClass\[] | | | File Stats | Performance statistics for slowest files | Debug rule performance | `[{"size": 6725,"numTimesScanned": 147,"parseTime": 0.013289928436279297,"matchTime": 0.05480456352233887,"runTime": 0.20836973190307617}]` | StatsClass\[] | | | | | | | | | | | | | | | | Parsing | | | | | | | | Total Files | Count of files, on a per-language basis | Understand parsing performance | 143 | Number | | | Total Bytes | Summation of target file size, likewise grouped | Understand parsing performance | 41244 | Number | | | Parsed Files | Count of files without parse errors | Understand parsing performance | 140 | Number | | | Parsed Bytes | Count of bytes without any parse errors | Understand parsing performance | 40312 | Number | | | | | | | | | Errors | | | | | | | | Exit Code | Numeric exit code | Debug commonly occurring issues and aggregate error counts | 1 | Number | | | Number of Errors | Count of errors | Understanding avg #errors | 2 | Number | | | Number of Warnings | Count of warnings | Understanding avg #warnings | 1 | Number | | | Errors | Array of Error Classes (compile-time-constant) | Understand most common errors users encounter | `["UnknownLanguage", "MaxFileSizeExceeded"] ` | ErrorClass\[] | | | Warnings | Array of Warning Classes (compile-time-constant) | Understand most common warnings users encounter | `["TimeoutExceeded"]` | WarningClass\[] | | | | | | | | | Value | | | | | | | | Engine requested | The engine type requested by the user | Understand which engines are being used; debug engine-specific problems | `"OSS"` | | | | Engine configuration | The specific engine configuration | Understand which engines are being used; debug engine-specific problems | `{ analysis_type: "Interfile", pro_langs: true, code_config: {} }` | str | | | Interfile languages used | The languages for which the interfile engine was actually invoked | Understand which interfile languages are being used; measure performance impact and errors | `["C#"]` | str | | | [Features used](#feature-usage) | List of strings that identify Semgrep features used | Understand what features users find valuable, and what we could deprecate | `["language/python", "option/deep", "option/no-git-ignore", "key/metavariable-comparison"]` | Object | | | Rule hashes with findings | Map of rule hashes to number of findings | Understand which rules are providing value to the user; diagnose high false-positive rates | `{"7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166": 4}` | Object | | | Total Findings | Count of all findings | Understand if rules are super noisy for the user | 7 | Number | | | Findings per product | Count of findings broken down by product | Understand the value that each product provides to the user | `{"code": 5, "secrets": 7, "supply-chain": 10}` | Object | | | Total Nosems | Count of all `nosem` annotations that tell semgrep to ignore a finding | Understand if rules are super noisy for the user | 3 | Number | | | | | | | | |Extension|||||| \| |Machine ID|A random UUID generated by the IDE itself|Understanding number of unique users using IDE integrations|`222bcccd-9dc2-4d10-ac3a-5692460e77ee`|String| \| |Is New App Install|If the user just installed the IDE integration|Understand common issues with setting up IDE integrations|`false`|Boolean| \| |Session ID|A random UUID generated everytime the integration starts up, usually when opening a project|Understand errors that commonly happen together, deduplicate errors|`222bcccd-9dc2-4d10-ac3a-5692460e77ee`|String| \| |Integration version|Current version of the IDE integration|Reproduce and debug issues with specific versions|`1.8.0`| String| \| |Integration type|IDE being used|Reproduce and debug issues with specific integrations|`vscode`|String| \| |Autofix count|How many autofixes have been triggered through the integration|Understand the value that the integration provides to the user in helping remediate code issue|10|Number| \| |Ignore count|How many findings have been ignored by the user through the integration|Understand the quality and noisiness of rules|5|Number| |Pro Engine install|||||| \| |Success|Whether `semgrep install-semgrep-pro` completed successfully|Understand how many Pro Engine installs happen and how many succeed|`true`|Boolean| \| |Error|Fixed identifier of the install step that failed|Debug common installation failures|`download-403`|String| ### Anonymous user ID > `anonymous_user_id: "5f52484c-3f82-4779-9353-b29bbd3193b6"` To help improve Semgrep products, the Semgrep CLI generates a Universally Unique Identifier (UUID) which is saved locally to a `~/.semgrep/settings.yml` file when the ID does not already exist. The Semgrep team uses this ID to help answer the following questions: * > How many people use a given rule/ruleset/snippet? This enables the Semgrep team to assess their performance, and we're planning to make these numbers public for all rule authors in the community. * > What percentage of users log in? We use this to evaluate our success as we build new authenticated features for the Semgrep Cloud Platform. * > How often are individual subcommands and CLI features used? This helps our product and developer experience teams measure feature adoption rate, analyze anonymized usage, and compare cohort behavior to improve our product offerings. ### Feature usage > `"features": ["language/python", "option/deep", "option/no-git-ignore", "key/metavariable-comparison"]` Examples of such features are: languages scanned, CLI options passed, keys used in rules, or certain code paths reached, such as using an :include instruction in a .semgrepignore file. These strings do NOT include user data or specific settings. As an example, for `semgrep scan --output=secret.txt` Semgrep sends `"option/output"` but will NOT send `"option/output=secret.txt"`. The list of features tracked as of June 2022 is: * `language`: What languages were scanned * `cli-flag`/`cli-envvar`: What options were configured (does NOT include their value) * `config`: What method was used to retrieve rules (does NOT include any of the rule) * `registry-query`: The value of a `--config r/foo.bar.baz` setting, limited to the first word (e.g. `r/foo..` in this example) * `ruleset`: The value of a `--config p/foobar` setting * `semgrepignore`: Whether an `:include` statement was used in a .semgrepignore file * `subcommand`: What subcommand was used (e.g. `scan` or `ci`) The Semgrep team uses this to answer the following questions: * > How many people use a given feature? This guides our development, and lets us decide when and how to deprecate features. * > How does feature usage affect finding counts, error counts, and performance? We use this to evaluate experimental features and understand their production-readiness. > Engine requested (OSS, Pro, Interfile) The engine requested is stored separately from the other features. This is the engine indicated by the user through app toggles or CLI flags. We use this for debugging as well as to understand which engines people are using. ### Sample metrics This is a sample blob of the aggregate metrics described above: ``` { "started_at": "2021-05-10T21:05:06+00:00", "sent_at": "2021-05-10T21:05:09+00:00", "event_id": "222bcccd-9dc2-4d10-ac3a-5692460e77ee", "anonymous_user_id": "5f52484c-3f82-4779-9353-b29bbd3193b6", "environment": { "version": "0.51.0", "ci": "true", "configNamesHash": "ede96c41b57de3e857090fb3c486e69ad8efae3267bac4ac5fbc19dde7161094", "projectHash": "c65437265631ab2566802d4d90797b27fbe0f608dceeb9451b979d1671c4bc1a", "rulesHash": "b03e452f389e5a86e56426c735afef13686b3e396499fc3c42561f36f6281c43", "isAuthenticated": false }, "performance": { "runTime": 37.1234233823, "numRules": 2, "numTargets": 573, "totalBytesScanned": 33938923, "ruleStats": [{ "ruleHash": "7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166", "parseTime": 0, "matchTime": 0.05480456352233887, "runTime": 0.20836973190307617, "bytesScanned": 0 }], "fileStats": [{ "size": 6725, "numTimesScanned": 147, "parseTime": 0.013289928436279297, "matchTime": 0.05480456352233887, "runTime": 0.20836973190307617 }] }, "parse_rate": { "python": { "num_targets": 102, "targets_parsed": 101, "num_bytes": 985123, "bytes_parsed": 993419 }, "ruby": { "num_targets": 12, "targets_parsed": 12, "num_bytes": 341027, "bytes_parsed": 341027 } }, "errors": { "returnCode": 1, "errors": ["UnknownLanguage"], "warnings": ["MaxFileSizeExceeded", "TimeoutExceeded"] }, "value": { "ruleHashesWithFindings": {"7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166": 4}, "numFindings": 7, "numIgnored": 3, "features": ["language/python", "option/deep", "option/no-git-ignore", "key/metavariable-comparison"], "engineRequested": "OSS", "engineConfig": { analysis_type: "Intraprocedural", pro_langs: false } } } ``` ## Data collected when explicitly requested For Semgrep AppSec Platform users running `semgrep ci` while logged in, data is sent to power your dashboard, notification, dependency search, and finding management features. These data are ONLY sent when using `semgrep ci` in a platform-connected mode and are not sent when not logged in. Three types of data are sent to Semgrep, Inc servers for this logged-in use case: scan data, findings data, and dependencies data. **Scan data** provide information on the environment and performance. They power dashboards, identify anomalies with the product, and are needed for billing. The classes of scan data are: * Project identity (e.g., name, URL) * Scan environment (e.g., CI provider, OS) * Author identity (e.g., committer email) * Commit metadata (e.g., commit hash and timestamp) * Review and review-requester identifying data (e.g., pull-request ID, branch, merge base, request author) * Scan metadata, including type of scan and scan parameters (e.g., paths scanned and extensions of ignored files) * Timing metrics (e.g., time taken to scan per-rule and per-path) * Parse metrics (e.g., number of files targeted and parsed per-language) * Semgrep CLI environment (e.g., version, interpreter, timestamp) **Findings data** are used to provide human readable content for notifications and integrations, as well tracking results as new, fixed, or duplicate. The classes of findings data are: * Check ID and metadata (as defined in the rule definition; e.g., OWASP category, message, severity) * Code location, including file path, that triggered findings * A one-way hash of a unique code identifier that includes the triggering code content * Dependency name and version (only sent when using Semgrep Supply Chain) * Source code is NOT collected **Dependencies data** are used to power Dependency Search and License Compliance. The classes of dependencies data are: * Package name (e.g., lodash) * Package version (e.g., 1.2.3) * File path for lockfile (e.g., frontend/yarn.lock) * Analysis of external dependency calls. (e.g., from flask import Response, Response(status=204)) ## Debugging data collected when traces are requested To help debug performance issues, Semgrep CLI can send traces, enabled via `--trace`. Traces are never sent unless the `--trace` flag is included, and are never retained for more than 30 days. There are three modes of tracing. 1. Info (`--trace`): basic tracing. Sends basic info about settings passed via environment variables or flags, and timings around scan phases. 2. Debug (`--trace` with `SEMGREP_TRACE_LEVEL=debug`): debug tracing. Sends additional timings, particularly around functions run during taint analysis, timings about each file as it undergoes pre-processing and then matching. Includes the file path and sometimes rule names. 3. Trace (`--trace` with `SEMGREP_TRACE_LEVEL=trace`): even more detailed debug tracing. All traces are sent in Opentelemetry format and may include: * Semgrep function currently running (e.g. `Match_tainting_mode.check_rules`) * Start time (e.g. `1718775054055113`) * Duration (e.g. `934956`) * Settings passed (e.g. `--jobs`, `--timeout`) Additionally, summary data is always included in the top level trace, such as: * Repo name (e.g. `semgrep-app`) * Number of matches (e.g. `2`) * Number of errors (e.g. `1`) * Number of rules (e.g. `12`) * Number of targets (e.g. `128`) * Is a diff scan (e.g. `false`) * Is an interfile scan (e.g. `true`) * Cryptographic hash of rules (e.g. `One-way hash of the rule definitions`) * Other scan settings of a similar nature. Summary data will only include information that `semgrep ci` has access to. Additionally, informational, warning, and error logs will be included when tracing is enabled, which may include: * Stacktraces from when Semgrep crashes * Warnings about high memory usage * Informational logs about which stage of scanning Semgrep is performing No information will be sent in the info mode that would not be sent by `semgrep ci`. In debug and trace mode only, traces may also include: * file paths (e.g. `src/my_file.py`) * Hashed function names (e.g. `d40fdc8ef9bf7b7dd1b014533a58a05e9b98d7dd856784352201388fe5e22673`) * Hashed variable names (e.g. `0268934f5c43d1b5fc7d52d9efe17c69f1144b108c384c3513cbe493043712b3`) These data help us establish if a function is being analyzed for taint more times than expected. Debug and trace mode are meant for one-off debugging of slow scans, and data from these trace modes will not be retained for more than a week. ## Registry fetches Certain Registry resources require log-in to the Semgrep Registry. Log in may be performed using your project URL, or a Semgrep.dev API token. When using these resources, your project's identity will be recorded by the Semgrep Registry servers. ## Data sharing We use some third party companies and services to help administer and provide Semgrep, for example for hosting, customer support, product usage analytics, and database management. These third parties are permitted to handle data only to perform these tasks in a manner consistent with this document and are obligated not to disclose or use it for any other purpose. We do not share or sell the information provided to us with other organizations without explicit consent, except as described in this document. # Semgrep metrics Source: https://docs.semgrep.dev/metrics-1 Semgrep CLI may collect aggregate metrics to help improve the product. This document describes: * [the principles that guide our data-collection decisions](#principles) * [how to change when Semgrep sends metrics](#automatic-collection-opt-in-and-opt-out) * [what data is not collected](#data-not-collected) * [what data is collected](#data-collected-as-metrics) # Semgrep metrics Semgrep CLI may collect aggregate metrics to help improve the product. This document describes: * [the principles that guide our data-collection decisions](#principles) * [how to change when Semgrep sends metrics](#automatic-collection-opt-in-and-opt-out) * [what data is not collected](#data-not-collected) * [what data is collected](#data-collected-as-metrics) ## Principles These principles inform our decisions around data collection: 1. **Transparency**: Collect and use data in a way that is clearly explained to the user and benefits them 2. **User control**: Put users in control of their data at all times 3. **Limited data**: Collect what is needed, pseudoanonymize where possible, and delete when no longer necessary ## Automatic collection, opt-in, and opt-out ```sh theme={null} $ semgrep --config=myrule.yaml # β no metrics (loading rules from local file) $ semgrep --config=p/python # β metrics enabled (fetching Registry) $ semgrep login && semgrep ci # β metrics enabled (logged in to semgrep.dev) $ semgrep install-semgrep-pro # β metrics enabled (requires being logged in) ``` Semgrep does **not** enable metrics when running with only local configuration files or command-line search patterns. Semgrep does enable metrics if rules are loaded from the [Semgrep Registry](https://semgrep.dev/r). This helps maintainers improve the correctness and performance of registry rules. Metrics may also be configured to be sent on every run, or never sent. To configure metrics, pass the `--metrics` option to Semgrep: * `--metrics auto`: (default) metrics are sent whenever rules are pulled from the [Semgrep Registry](https://semgrep.dev/r) or the user is logged in. * `--metrics on`: metrics are sent on every Semgrep run * `--metrics off`: metrics are never sent Alternatively, set the `SEMGREP_SEND_METRICS` environment variable to `auto`, `on`, or `off`. Note that certain Semgrep integrators turn on metrics for every run. For example, [GitLab's Semgrep SAST analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) uses `--metrics on` by default. ## Data NOT collected ### Data NOT collected ever We strive to balance our desire to collect data for improving Semgrep with our users' need for privacy and security. After all, we are a security tool! The following never leave your environment and are not sent or shared with anyone. * Source code * Private rules ### Data NOT collected unless explicitly requested The following data will never leave your environment as part of metrics. * Filenames * Git commit hashes, timestamps, messages, authors * User-identifiable data about Semgrepβs findings in your code, including finding messages This data will be sent to Semgrep AppSec Platform only if you explicitly request it, such as with `semgrep login && semgrep ci` to connect with Semgrep AppSec Platform. Even in that case, your source code and private rules will never be sent. ## Data collected as metrics Semgrep CLI collects data to improve the user experience. Five types of data are collected: ### Environmental Environmental data provide contextual data about Semgrep CLIβs runtime environment, as well as information that helps debug any issues users may be facing; e.g. * How long the command took to run * The version of Semgrep CLI * An [anonymous user ID](#anonymous-user-id) that identifies the machine * IP address that triggers a run * Value of the CI environment variable, if set * How the Semgrep CLI was installed (`pip`, `homebrew`, `docker`, or `unknown`), detected heuristically from the install path and environment variables * Pseudoanonymized hash of the scanned projectβs name * Pseudoanonymized hash of the rule definitions run * Pseduoanonymized hash of the config option. *(Note that when a config option downloads a ruleset from the [https://semgrep.dev](https://semgrep.dev) registry, [feature usage metrics](#feature-usage) still include the ruleset name in plain text.)* ### Performance Performance data enable understanding of which rules and types of files are slow in the aggregate so Semgrep, Inc can improve the program-analysis engine, query optimizer, and debug slow rules; e.g. * Runtime duration * Duration of individual phases (e.g. parsing) * Total number of rules * Total number of files * Project size in bytes ### Parse Rates Aggregated parse rate information is reported on a per-language basis; e.g., * Number of targeted files * Number of files without any parse-related error * Number of bytes across targeted files * Number of bytes without any parse-related error ### Errors High-level error and warning classes encountered when run; e.g. * Semgrepβs return code * The number of errors * Compile-time error names, e.g., MaxFileSizeExceeded, SystemOutOfMemory, UnknownFileEncoding ### Value Data that indicate how useful a run is for the end user; e.g. * Number of raised findings * Number of ignored findings * Pseudoanonymized hashes of the rule definitions that yield findings * The [features used](#feature-usage) during the scan * The engine type requested for the scan ### Extension Additional data is reported when used in conjunction with an IDE integration, such as the [Semgrep VS Code Extension](https://github.com/semgrep/semgrep-vscode), that help us understand what IDEs are used and how helpful the integrations are for users; e.g. * IDE being used * Version of IDE integration * Number of fixes triggered through the integration * Number of findings ignored through the integration Note: For all officially supported Semgrep IDE integrations, these metrics can be disabled via settings in the IDE. By default these settings follow any global telemetry/metrics settings the user may have already set for the IDE itself. ### Pro Engine installation When running `semgrep install-semgrep-pro` (which requires being logged in), additional data is reported to help us understand how many Pro Engine installations happen and how many succeed; e.g. * Whether the installation completed successfully * Which installation step failed, as a fixed identifier such as `download-403` or `version-check-failed` (never a free-form error message) ### Pseudoanonymization Certain identifying data (e.g. project URLs) are pseudoanonymized before being sent to the Semgrep, Inc backend. "Pseudoanonymized" means the data are transformed using a deterministic cryptographically secure hash. When the input data are unknown, this hash is expensive to reverse. However, when input data are known, a reverse dictionary of identifiers to hashes can be built. Hence, data are anonymous only when the source values are unknown. We use a deterministic hash to: * Track performance and value improvements over successive runs on projects * Remove test data from our metrics Using a deterministic hash, however, implies: * An entity that independently knows the value of an input datum AND who has access to Semgrep, Inc's metrics data could access metrics for that known datum Semgrep, Inc will: * Treat collected metrics data as secret, using application-security best practices, including (but not limited to) * Encryption during transit and rest * Strict access control to data-storage systems * Application-security-policy requirements for third parties (e.g. cloud-service providers; see "data sharing" below) * Only correlate hashed data to input data when these inputs are already known to Semgrep, Inc (e.g. publicly available project URLs for open-source projects, or projects that log in to the Semgrep Registry) ## Description of metrics fields | Category | Field | Description | Use Case | Example Datum | Type | | ----------- | --------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | | Environment | | | | | | | | Timestamps (started/sent) | Time when the event fired | Understanding tool usage over time | 2021-05-10T21:05:06+00:00 | String | | | Event ID | A random UUID generated when sending the event. | Deduplicating events in case of issues during transmission | 222bcccd-9dc2-4d10-ac3a-5692460e77ee | String | | | [Anonymous User ID](#anonymous-user-id) | A random UUID generated on first run. | Unique users per ruleset and feature. Understanding percentage of logged in users. | 5f52484c-3f82-4779-9353-b29bbd3193b6 | String | | | Version | Semgrep version being used | Reproduce and debug issues with specific versions | 0.51.0 | String | | | Project hash | One-way hash of the project URL | Understand performance and accuracy improvements | `c65437265631ab2566802d4d90797b27fbe0f608dceeb9451b979d1671c4bc1a` | String | | | Rules hash | One-way hash of the rule definitions | Understand performance improvements | `b03e452f389e5a86e56426c735afef13686b3e396499fc3c42561f36f6281c43` | String | | | Config hash | One-way hash of the config argument | Understand performance and accuracy improvements | `ede96c41b57de3e857090fb3c486e69ad8efae3267bac4ac5fbc19dde7161094` | String | | | Is authenticated | Whether the user logged in to semgrep.dev with `semgrep login` | Understand popularity of logged in features | `false` | Boolean | | | Deployment ID | The ID organization associated with the logged in account | Understand popularity of logged in features by organization | 1234 | Number | | | Integration name | If Semgrep is being called by another tool, optional name of that integration | Reproduce and debug issues specific integrations | `gitlab` | String | | | CI | Notes if Semgrep is running in CI and the name of the provider | Reproduce and debug issues with specific CI providers | GitLabCI v0.13.12 | String | | | Install method | How the Semgrep CLI was installed, detected heuristically | Understand which distribution channels are used; debug channel-specific issues | `pip` | String | | | Client IP | IP address that triggered a run | Understand broad ruleset usage | 0.0.0.0 | String | | | | | | | | | Performance | | | | | | | | Duration | How long the command took to run | Understanding aggregate performance improvements and regressions | 14.13 | Number | | | Total Rules | Count of rules | Understand how duration is affected by #rules | 137 | Number | | | Total Files | Count of files | Understand how duration is affected by #files | 4378 | Number | | | Total Bytes | Summation of target file size | Understand how duration is related to total size of all target files | 40838239 | Number | | | Rule Stats | Performance statistics (w/ rule hashes) for slowest rules | Debug rule performance | `[{"ruleHash": "7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166","parseTime": 0,"matchTime": 0.05480456352233887,"runTime": 0.20836973190307617,"bytesScanned": 0}]` | StatsClass\[] | | | File Stats | Performance statistics for slowest files | Debug rule performance | `[{"size": 6725,"numTimesScanned": 147,"parseTime": 0.013289928436279297,"matchTime": 0.05480456352233887,"runTime": 0.20836973190307617}]` | StatsClass\[] | | | | | | | | | | | | | | | | Parsing | | | | | | | | Total Files | Count of files, on a per-language basis | Understand parsing performance | 143 | Number | | | Total Bytes | Summation of target file size, likewise grouped | Understand parsing performance | 41244 | Number | | | Parsed Files | Count of files without parse errors | Understand parsing performance | 140 | Number | | | Parsed Bytes | Count of bytes without any parse errors | Understand parsing performance | 40312 | Number | | | | | | | | | Errors | | | | | | | | Exit Code | Numeric exit code | Debug commonly occurring issues and aggregate error counts | 1 | Number | | | Number of Errors | Count of errors | Understanding avg #errors | 2 | Number | | | Number of Warnings | Count of warnings | Understanding avg #warnings | 1 | Number | | | Errors | Array of Error Classes (compile-time-constant) | Understand most common errors users encounter | `["UnknownLanguage", "MaxFileSizeExceeded"] ` | ErrorClass\[] | | | Warnings | Array of Warning Classes (compile-time-constant) | Understand most common warnings users encounter | `["TimeoutExceeded"]` | WarningClass\[] | | | | | | | | | Value | | | | | | | | Engine requested | The engine type requested by the user | Understand which engines are being used; debug engine-specific problems | `"OSS"` | | | | Engine configuration | The specific engine configuration | Understand which engines are being used; debug engine-specific problems | `{ analysis_type: "Interfile", pro_langs: true, code_config: {} }` | str | | | Interfile languages used | The languages for which the interfile engine was actually invoked | Understand which interfile languages are being used; measure performance impact and errors | `["C#"]` | str | | | [Features used](#feature-usage) | List of strings that identify Semgrep features used | Understand what features users find valuable, and what we could deprecate | `["language/python", "option/deep", "option/no-git-ignore", "key/metavariable-comparison"]` | Object | | | Rule hashes with findings | Map of rule hashes to number of findings | Understand which rules are providing value to the user; diagnose high false-positive rates | `{"7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166": 4}` | Object | | | Total Findings | Count of all findings | Understand if rules are super noisy for the user | 7 | Number | | | Findings per product | Count of findings broken down by product | Understand the value that each product provides to the user | `{"code": 5, "secrets": 7, "supply-chain": 10}` | Object | | | Total Nosems | Count of all `nosem` annotations that tell semgrep to ignore a finding | Understand if rules are super noisy for the user | 3 | Number | | | | | | | | |Extension|||||| \| |Machine ID|A random UUID generated by the IDE itself|Understanding number of unique users using IDE integrations|`222bcccd-9dc2-4d10-ac3a-5692460e77ee`|String| \| |Is New App Install|If the user just installed the IDE integration|Understand common issues with setting up IDE integrations|`false`|Boolean| \| |Session ID|A random UUID generated everytime the integration starts up, usually when opening a project|Understand errors that commonly happen together, deduplicate errors|`222bcccd-9dc2-4d10-ac3a-5692460e77ee`|String| \| |Integration version|Current version of the IDE integration|Reproduce and debug issues with specific versions|`1.8.0`| String| \| |Integration type|IDE being used|Reproduce and debug issues with specific integrations|`vscode`|String| \| |Autofix count|How many autofixes have been triggered through the integration|Understand the value that the integration provides to the user in helping remediate code issue|10|Number| \| |Ignore count|How many findings have been ignored by the user through the integration|Understand the quality and noisiness of rules|5|Number| |Pro Engine install|||||| \| |Success|Whether `semgrep install-semgrep-pro` completed successfully|Understand how many Pro Engine installs happen and how many succeed|`true`|Boolean| \| |Error|Fixed identifier of the install step that failed|Debug common installation failures|`download-403`|String| ### Anonymous user ID > `anonymous_user_id: "5f52484c-3f82-4779-9353-b29bbd3193b6"` To help improve Semgrep products, the Semgrep CLI generates a Universally Unique Identifier (UUID) which is saved locally to a `~/.semgrep/settings.yml` file when the ID does not already exist. The Semgrep team uses this ID to help answer the following questions: * > How many people use a given rule/ruleset/snippet? This enables the Semgrep team to assess their performance, and we're planning to make these numbers public for all rule authors in the community. * > What percentage of users log in? We use this to evaluate our success as we build new authenticated features for the Semgrep Cloud Platform. * > How often are individual subcommands and CLI features used? This helps our product and developer experience teams measure feature adoption rate, analyze anonymized usage, and compare cohort behavior to improve our product offerings. ### Feature usage > `"features": ["language/python", "option/deep", "option/no-git-ignore", "key/metavariable-comparison"]` Examples of such features are: languages scanned, CLI options passed, keys used in rules, or certain code paths reached, such as using an :include instruction in a .semgrepignore file. These strings do NOT include user data or specific settings. As an example, for `semgrep scan --output=secret.txt` Semgrep sends `"option/output"` but will NOT send `"option/output=secret.txt"`. The list of features tracked as of June 2022 is: * `language`: What languages were scanned * `cli-flag`/`cli-envvar`: What options were configured (does NOT include their value) * `config`: What method was used to retrieve rules (does NOT include any of the rule) * `registry-query`: The value of a `--config r/foo.bar.baz` setting, limited to the first word (e.g. `r/foo..` in this example) * `ruleset`: The value of a `--config p/foobar` setting * `semgrepignore`: Whether an `:include` statement was used in a .semgrepignore file * `subcommand`: What subcommand was used (e.g. `scan` or `ci`) The Semgrep team uses this to answer the following questions: * > How many people use a given feature? This guides our development, and lets us decide when and how to deprecate features. * > How does feature usage affect finding counts, error counts, and performance? We use this to evaluate experimental features and understand their production-readiness. > Engine requested (OSS, Pro, Interfile) The engine requested is stored separately from the other features. This is the engine indicated by the user through app toggles or CLI flags. We use this for debugging as well as to understand which engines people are using. ### Sample metrics This is a sample blob of the aggregate metrics described above: ``` { "started_at": "2021-05-10T21:05:06+00:00", "sent_at": "2021-05-10T21:05:09+00:00", "event_id": "222bcccd-9dc2-4d10-ac3a-5692460e77ee", "anonymous_user_id": "5f52484c-3f82-4779-9353-b29bbd3193b6", "environment": { "version": "0.51.0", "ci": "true", "configNamesHash": "ede96c41b57de3e857090fb3c486e69ad8efae3267bac4ac5fbc19dde7161094", "projectHash": "c65437265631ab2566802d4d90797b27fbe0f608dceeb9451b979d1671c4bc1a", "rulesHash": "b03e452f389e5a86e56426c735afef13686b3e396499fc3c42561f36f6281c43", "isAuthenticated": false }, "performance": { "runTime": 37.1234233823, "numRules": 2, "numTargets": 573, "totalBytesScanned": 33938923, "ruleStats": [{ "ruleHash": "7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166", "parseTime": 0, "matchTime": 0.05480456352233887, "runTime": 0.20836973190307617, "bytesScanned": 0 }], "fileStats": [{ "size": 6725, "numTimesScanned": 147, "parseTime": 0.013289928436279297, "matchTime": 0.05480456352233887, "runTime": 0.20836973190307617 }] }, "parse_rate": { "python": { "num_targets": 102, "targets_parsed": 101, "num_bytes": 985123, "bytes_parsed": 993419 }, "ruby": { "num_targets": 12, "targets_parsed": 12, "num_bytes": 341027, "bytes_parsed": 341027 } }, "errors": { "returnCode": 1, "errors": ["UnknownLanguage"], "warnings": ["MaxFileSizeExceeded", "TimeoutExceeded"] }, "value": { "ruleHashesWithFindings": {"7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166": 4}, "numFindings": 7, "numIgnored": 3, "features": ["language/python", "option/deep", "option/no-git-ignore", "key/metavariable-comparison"], "engineRequested": "OSS", "engineConfig": { analysis_type: "Intraprocedural", pro_langs: false } } } ``` ## Data collected when explicitly requested For Semgrep AppSec Platform users running `semgrep ci` while logged in, data is sent to power your dashboard, notification, dependency search, and finding management features. These data are ONLY sent when using `semgrep ci` in a platform-connected mode and are not sent when not logged in. Three types of data are sent to Semgrep, Inc servers for this logged-in use case: scan data, findings data, and dependencies data. **Scan data** provide information on the environment and performance. They power dashboards, identify anomalies with the product, and are needed for billing. The classes of scan data are: * Project identity (e.g., name, URL) * Scan environment (e.g., CI provider, OS) * Author identity (e.g., committer email) * Commit metadata (e.g., commit hash and timestamp) * Review and review-requester identifying data (e.g., pull-request ID, branch, merge base, request author) * Scan metadata, including type of scan and scan parameters (e.g., paths scanned and extensions of ignored files) * Timing metrics (e.g., time taken to scan per-rule and per-path) * Parse metrics (e.g., number of files targeted and parsed per-language) * Semgrep CLI environment (e.g., version, interpreter, timestamp) **Findings data** are used to provide human readable content for notifications and integrations, as well tracking results as new, fixed, or duplicate. The classes of findings data are: * Check ID and metadata (as defined in the rule definition; e.g., OWASP category, message, severity) * Code location, including file path, that triggered findings * A one-way hash of a unique code identifier that includes the triggering code content * Dependency name and version (only sent when using Semgrep Supply Chain) * Source code is NOT collected **Dependencies data** are used to power Dependency Search and License Compliance. The classes of dependencies data are: * Package name (e.g., lodash) * Package version (e.g., 1.2.3) * File path for lockfile (e.g., frontend/yarn.lock) * Analysis of external dependency calls. (e.g., from flask import Response, Response(status=204)) ## Debugging data collected when traces are requested To help debug performance issues, Semgrep CLI can send traces, enabled via `--trace`. Traces are never sent unless the `--trace` flag is included, and are never retained for more than 30 days. There are three modes of tracing. 1. Info (`--trace`): basic tracing. Sends basic info about settings passed via environment variables or flags, and timings around scan phases. 2. Debug (`--trace` with `SEMGREP_TRACE_LEVEL=debug`): debug tracing. Sends additional timings, particularly around functions run during taint analysis, timings about each file as it undergoes pre-processing and then matching. Includes the file path and sometimes rule names. 3. Trace (`--trace` with `SEMGREP_TRACE_LEVEL=trace`): even more detailed debug tracing. All traces are sent in Opentelemetry format and may include: * Semgrep function currently running (e.g. `Match_tainting_mode.check_rules`) * Start time (e.g. `1718775054055113`) * Duration (e.g. `934956`) * Settings passed (e.g. `--jobs`, `--timeout`) Additionally, summary data is always included in the top level trace, such as: * Repo name (e.g. `semgrep-app`) * Number of matches (e.g. `2`) * Number of errors (e.g. `1`) * Number of rules (e.g. `12`) * Number of targets (e.g. `128`) * Is a diff scan (e.g. `false`) * Is an interfile scan (e.g. `true`) * Cryptographic hash of rules (e.g. `One-way hash of the rule definitions`) * Other scan settings of a similar nature. Summary data will only include information that `semgrep ci` has access to. Additionally, informational, warning, and error logs will be included when tracing is enabled, which may include: * Stacktraces from when Semgrep crashes * Warnings about high memory usage * Informational logs about which stage of scanning Semgrep is performing No information will be sent in the info mode that would not be sent by `semgrep ci`. In debug and trace mode only, traces may also include: * file paths (e.g. `src/my_file.py`) * Hashed function names (e.g. `d40fdc8ef9bf7b7dd1b014533a58a05e9b98d7dd856784352201388fe5e22673`) * Hashed variable names (e.g. `0268934f5c43d1b5fc7d52d9efe17c69f1144b108c384c3513cbe493043712b3`) These data help us establish if a function is being analyzed for taint more times than expected. Debug and trace mode are meant for one-off debugging of slow scans, and data from these trace modes will not be retained for more than a week. ## Registry fetches Certain Registry resources require log-in to the Semgrep Registry. Log in may be performed using your project URL, or a Semgrep.dev API token. When using these resources, your project's identity will be recorded by the Semgrep Registry servers. ## Data sharing We use some third party companies and services to help administer and provide Semgrep, for example for hosting, customer support, product usage analytics, and database management. These third parties are permitted to handle data only to perform these tasks in a manner consistent with this document and are obligated not to disclose or use it for any other purpose. We do not share or sell the information provided to us with other organizations without explicit consent, except as described in this document. # Prerequisites Source: https://docs.semgrep.dev/prerequisites This document details the required software or services to run Semgrep products. ## Overall A programming language must be supported by Semgrep for your chosen product. | Product | Scan type | Link | | ------------------------------ | --------- | -------------------------------------------------------------------- | | Semgrep Community Edition (CE) | SAST | [Supported languages](/supported-languages#language-maturity-levels) | | Semgrep Code | SAST | [Supported languages](/supported-languages#language-maturity-levels) | | Semgrep Supply Chain | SCA | [Supported languages](/supported-languages#semgrep-supply-chain) | | Semgrep Secrets | Secrets | Language-agnostic | ## Semgrep command-line tool These requirements apply to both Semgrep AppSec Platform and Semgrep CE. ### Software * Python 3.10 or later installed on the machine you are running Semgrep on. ### Operating system * macOS * Linux * Windows (beta) ## Semgrep AppSec Platform These requirements apply to Semgrep AppSec Platform. * A GitHub or GitLab cloud account. The credentials are used to authenticate and identify you. * A Git repository to scan, stored in any of the following source code managers: * GitHub * GitLab * Bitbucket * Azure DevOps * A CI provider and sufficient permissions to create CI jobs. # Feature definitions Source: https://docs.semgrep.dev/references/feature-definitions This document defines the terms used when discussing Semgrep analysis features in [Supported languages](/supported-languages). ## Cross-file dataflow analysis Cross-file analysis (also known as **interfile analysis**) takes into account how information flows between files. In particular, cross-file analysis includes **cross-file taint analysis**, which tracks unsanitized variables flowing from a source to a sink through arbitrarily many files. Other analyses performed across files include constant propagation and type inference. Cross-file analysis is usually used in contrast to intrafile, or per-file analysis, where each file is analyzed as a standalone block of code. Languages with cross-file support also include cross-function support. ## Cross-function dataflow analysis Cross-function analysis means that interactions between functions are taken into account. This improves taint analysis, which tracks unsanitized variables flowing from a source to a sink through arbitrarily many functions. ## Reachability analysis Reachability refers to whether or not a vulnerable code pattern from a dependency is used in the codebase that imports it. In Semgrep Supply Chain, both a dependency's vulnerable version and code pattern must match for a vulnerability to be considered reachable. See [Overview of Semgrep Supply Chain](/semgrep-supply-chain/overview) to learn how Semgrep leverages its code-scanning and rule syntax capabilities to provide high-signal rules that determine a finding's reachability. This assists security engineers in remediation and triage processes. # Language maturity levels Source: https://docs.semgrep.dev/references/language-maturity-levels This document defines the language maturity levels used on the [Supported languages](/supported-languages) page. ## Semgrep Code Semgrep Code languages can be classified into four maturity levels: * Generally available (GA) * Beta * Experimental * Community supported\* \*Community supported languages meet the parse rate and syntax requirements of **Experimental** languages. Users can still access community rules or write their own rules. | Feature | GA | Beta | Experimental | Community supported | | :------------------ | :----------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------- | | Support | Highest quality support by the Semgrep team. Reported issues are resolved promptly. | Supported by the Semgrep team. Reported issues are fixed after GA languages. | There are limitations to this language's functionality. Reported issues are tracked and prioritized with best effort. | These languages are supported by the Semgrep community. While Semgrep may develop rules or engine updates for these languages, they are not prioritized. | | Parse Rate | 99%+ | 95%+ | 90%+ | 90%+ | | Number of Pro rules | 10+ | 5+ | 0+. Query the [Registry](https://semgrep.dev/r) to see if any rules exist for your language. | 0+. Query the [Registry](https://semgrep.dev/r) to see if any rules exist for your language. | | Semgrep syntax | Regex, equivalence, deep expression operators, types and typing. All features supported in Beta. | Complete metavariable support, metavariable equality. All features supported in Experimental. | Syntax, ellipsis operator, basic metavariable functionality. | Syntax, ellipsis operator, basic metavariable functionality. | ## Semgrep Supply Chain Semgrep Supply Chain has two language maturity levels: * Generally available * Beta | Feature | Generally available | Beta | | :--------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Number of reachability rules | As defined by [CVE coverage](/semgrep-supply-chain/requirements-and-feature-support#cve-coverage). | All critical severity CVEs from [supported sources](/semgrep-supply-chain/requirements-and-feature-support#supported-sources) starting 2022 onwards, for packages used by customers with an active, paid subscription. | | Semgrep, Inc. rule-writing support | Quickly support CVE coverage with reachability analysis for all critical and high vulnerabilities based on the latest [security advisories](https://nvd.nist.gov/vuln). | Coverage for CVEs but without reachability analysis. | | [Semgrep Community Edition (CE) language support](/supported-languages#semgrep-oss-language-support) | Semgrep CE support is GA. | Semgrep CE support is at least Beta. | Not finding what you need in this doc? Ask questions in our [Community Slack group](https://go.semgrep.dev/slack), or see [Support](/support/) for other ways to get help. # Week of July 6, 2026 Source: https://docs.semgrep.dev/release-notes/2026-07-06 Updates made to Semgrep during the week of July 6-12, 2026. ## π Semgrep AppSec Platform ### Added * You can now archive a project from **Project settings**. Projects archived in Semgrep appear on an **Archived** tab with an archived badge, alongside projects archived in your source code manager. * You can configure Jira subtasks in the Jira integration settings. * The Policies automations condition builder menus now include typeahead search, so you can find condition types more quickly. * Remediation policy automations can now include a vulnerability class condition, so you can scope a policy to a class such as SQL Injection or Path Traversal. * **API**: Added public API endpoints to read, validate, dry-run, and apply detection and remediation policies for Code, Secrets, and Supply Chain. A related endpoint lists the accepted condition and action values. ### Changed * The **Dashboard's** **Secure guardrails** section is now **Developer engagement**, and its charts have updated titles. * Redesigned the contributor seats and AI credit usage cards on the **Usage & billing** page with a headline-led layout and updated status badges. * The **Findings** list now excludes archived issues by default. * Improved performance of the project scan page and Managed Scans search for large and highly active organizations. ### Fixed * Fixed an issue where audit events from background tasks were silently dropped instead of recorded. * Fixed an issue where expired license alerts on the **Usage & billing** page continued to display after live seats were available. * Fixed issues when migrating Slack notifications to unified policies: * Migrating legacy Slack notifications could drop Supply Chain notifications or misclassify a policy's product. * Migrating a project-scoped Slack subscription could create a global policy for every project instead of only the subscribed project. * Fixed an issue where Azure DevOps webhook validation called the wrong URL. ## π» Semgrep Code ### Changed * **Dart**: Updated the Dart parser to a more recent upstream version. ## βοΈ Semgrep Supply Chain ### Added * You can now update and enable an inactive Supply Chain incident policy in one step, instead of editing and then enabling separately. ### Fixed * Fixed an issue where the **Open Autofix PR** action was hidden for some organizations that use Upgrade guidance and Autofix separately. * Fixed an issue where the Supply Chain finding **Details** page timed out for findings with long dependency paths. ## π€ Semgrep Multimodal ### Fixed * Fixed an issue where AI-assisted scan submissions could bypass a customer's configured Amazon Bedrock opt-out. * Fixed permission errors when creating or listing organization-wide context documentation for AI-powered detection. * Fixed an issue where secrets in customer code, such as hardcoded credentials, could appear verbatim in saved AI remediation guidance. Secrets are now redacted before guidance is saved. * Fixed an issue where automatically creating a Memory from a bulk ignore-with-note action could apply to the wrong findings when multiple issue-state filters were selected. * Fixed an issue where **Suggested memories** pull request (PR) comment links pointed to the wrong PR and returned a 404 on GitHub. ## π Semgrep Secrets ### Fixed * Fixed an issue where generic secrets findings without an AI autotriage verdict did not appear in the Secrets findings list, even though their **Details** page rendered correctly. ## π§ Semgrep Community Edition The following versions of Semgrep Community Edition were released during the week of July 6-12, 2026:
i. Upgrade the dependency to a safe version.
ii. Lets the developer know if the upgrade is safe or if there are breaking changes and what those changes are. * **Transitive reachability** is now in **private beta**. For JavaScript projects, Semgrep reachability now extends to transitive dependencies. ### Changed * Increased the rate limit for SBOM exports through the Semgrep API. * Improved Supply Chain PR comments by adding separate templates for conditionally reachable and always reachable findings, as well as manual review advice for conditionally reachable findings. * Improved the user introduction to Supply Chain to focus on reachable findings. * Improved the **Supply Chain > Details** page. ## π€ Semgrep Assistant ### Added * Semgrep Assistant now attempts to create a memory during triage if possible. If Semgrep creates a memory, you'll see a dialog appear, indicating that this has happened, along with a link to the list of your organization's memories for review. * Assistant Memories v2 is now in **private beta**: * Managing memories in Semgrep AppSec Platform now occurs under **Policies**, not **Settings**. * Semgrep AppSec Platform displays data on the scope and impact of memories, including the number of findings affected and which findings affected * Assistant now provides **suggested memories**, which are those that Assistant has generated based on your past triage actions. You can view these memories at any time in Semgrep AppSec Platform by navigating to **Rules & Policies > Assistant Memories > Suggested**. For each suggestion, you can choose one of the following actions: * Activate the suggested memory to inform Assistant's future advice. * Edit the memory, then activate it. * Delete the memory. ## π Semgrep Secrets ### Fixed * Fixed an issue where Semgrep AppSec Platform didn't display the correct number of Secrets findings in the navigation bar. ## π Documentation and knowledge base ### Added * Semgrep release notes are now available through RSS. You can subscribe to the: * [
- Verify that your technical stack is supported by Semgrep.
- Begin gathering necessary permissions from your organization for **technical resources** to run the POV.
- Both parties establish success criteria and alignment of the POV goals through a **kickoff call**.
- Semgrep prepares for the POV by creating a dedicated Slack channel and other necessary accounts.
- Semgrep deployment rollout.
- Detection and remediation of findings.
- Analysis of Semgrep ROI.
- A roadmap call with the Semgrep product team.
- A rule-writing session where you can learn how to write custom Semgrep rules.
- Verify that your technical stack is supported by Semgrep.
- Begin gathering necessary permissions from your organization for **technical resources** to run the POV.
- Both parties establish success criteria and alignment of the POV goals through a **kickoff call**.
- Semgrep prepares for the POV by creating a dedicated Slack channel and other necessary accounts.
- Semgrep deployment rollout.
- Detection and remediation of findings.
- Analysis of Semgrep ROI.
- A roadmap call with the Semgrep product team.
- A rule-writing session where you can learn how to write custom Semgrep rules.
- `category`
- `confidence`
- `likelihood`
- `impact`
- `subcategory`
Read the [metadata reference documentation](/contributing/contributing-to-semgrep-rules-repository#fields-required-by-the-security-category). | | Verify that the rule matches as intended |
- See [Testing rules](/writing-rules/testing-rules).
- Enable [Code search (beta)](/semgrep-code/editor#code-search-beta) to test the rule on live repositories.
- [Ensure that PR or MR comments have been set up correctly.](/category/pr-or-mr-comments)
- Set the rule to [Comment or Block mode](/semgrep-code/policies#block-a-pr-or-mr-through-rule-modes).
For IDEs: Require developers to install the [Semgrep extension for their IDE](/extensions/overview).
For `pre-commit`: [Install and configure Semgrep for `pre-commit`](/extensions/pre-commit).
| ### Secure default snippet When creating a custom secure default, you must use `category: security` and `subcategory: secure default` values in your rule: ```yaml theme={null} rules: - id: some-custom-default ... metadata: category: security subcategory: - secure default ... ``` # Secure defaults Source: https://docs.semgrep.dev/secure-guardrails/secure-defaults **Secure defaults** are inherently secure libraries, frameworks, configurations, or settings. They mitigate common security concerns, such as preventing cross-site request forgery (CSRF) by properly verifying inbound requests in Django or Flask applications. By adopting secure defaults, teams minimize the need for developers to manually implement security measures. Secure default rules are Semgrep Code (SAST) rules that codify a secure default. The Semgrep team recommends deploying these rules as guardrails because the early adoption of secure defaults helps prevent additional vulnerabilities. Some secure default rules codify universally secure practices and work out of the box, while others are organization-specific and require customization. In the following example, the rule detects if a Flask [WTForm](https://flask-wtf.readthedocs.io/en/0.15.x/config/) view is protected from CSRF by default by checking the configuration variable `WTF_CSRF_CHECK_DEFAULT`. If it is set to `False` then the developer must call `csrf.protect()` whenever they handle a requestβa manual process they must remember every time. Thus, `WTF_CSRF_CHECK_DEFAULT=True` is a secure default, which this Semgrep rule enforces. ## Semgrep Code supported languages Semgrep Code provides secure default rules for the following languages: * C# * Python (Flask, FastAPI, and Django frameworks) Custom rules to deploy secure default rules can be written in any of [Semgrep Codeβs supported languages](/supported-languages). ## View Semgrep secure default rules View all proprietary Semgrep secure default rules through the ruleset [p/secure-defaults](https://semgrep.dev/p/secure-defaults). ## Next stepsThe content that identifies, explains, and provides remediation guidance for the security issue, such as the Semgrep rule and Semgrep Multimodal (AI) remediation guidance. Semgrep uses **rules**, which are instructions that detect patterns in your code, such as security issues, bugs, and more. Semgrep generates and reports **findings** to you whenever it finds code that matches the patterns defined by rules. Semgrep rules also include a **message** that guides remediation and provides other metadata about the vulnerability, such as its OWASP category, which are presented to the developer. Further improvements to this guidance are made through Semgrep Multimodal. **Interface**
The developer-native **interface** where the developer can see the content and triage or remediate the finding, such as Visual Studio Code (VS Code), pre-commit on CLI, or GitHub pull request comments. See [all supported interfaces](#support-for-developer-interfaces-pre-build).
- Receive human-written remediation guidance.
- Ignore or apply a [Rule-defined fix](/writing-rules/rule-defined-fix), if it is available, to findings individually.
- Receive human-written remediation guidance.
- Ignore or apply a [Rule-defined fix](/writing-rules/rule-defined-fix), if it is available, to findings individually.
- Receive human-written and Semgrep Multimodal remediation guidance\*; you can customize the confidence level at which the AI leaves a comment.
- Ignore or apply a [Rule-defined fix](/writing-rules/rule-defined-fix), if it is available, to findings individually.
- Receive human-written and Semgrep Multimodal remediation guidance\*; you can customize the confidence level at which the AI leaves a comment.
- Ignore or apply a [Rule-defined fix](/writing-rules/rule-defined-fix), if it is available, to findings individually.
- Receive human-written and Semgrep Multimodal remediation guidance\*; you can customize the confidence level at which the AI leaves a comment.
- Ignore or apply a [Rule-defined fix](/writing-rules/rule-defined-fix), if it is available, to findings individually.
- Receive human-written and Semgrep Multimodal remediation guidance\*; you can customize the confidence level at which the AI leaves a comment.
- Ignore or apply a [Rule-defined fix](/writing-rules/rule-defined-fix), if it is available, to findings individually.
- Receive human-written remediation guidance.
- Ignore or apply a [Rule-defined fix](/writing-rules/rule-defined-fix), if it is available, to findings individually.
1. [Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Code](#configure-comments-for-semgrep-code). | | Software composition analysis (SCA) | Semgrep Supply Chain (SSC) | A comment appears based on the conditions you explicitly set in a [Supply Chain policy](/semgrep-supply-chain/policies) or when Semgrep detects a [license violation](/semgrep-supply-chain/license-compliance). | To receive Supply Chain comments, complete the steps in [Confirm account connection and access](#confirm-your-semgrep-accounts-connection) and [set up a policy](/semgrep-supply-chain/policies).
To receive license violation comments, [enable dependency search](/semgrep-supply-chain/dependency-search#enable-and-use-dependency-search). | | Secrets | Semgrep Secrets | A comment appears when a finding is generated by a rule in **Comment or Block mode**. A comment also appears for invalid findings and validation errors if these conditions are set to **Comment or Block mode**. | Complete the steps in the following sections:
1. [Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Secrets](#configure-comments-for-semgrep-secrets). | Comments from Supply Chain scans include the following information: **Risk**
A description of the vulnerability, including the types of attack it is vulnerable to. **Fix**
Indicates what versions to upgrade to, if any, that resolves or eliminates the vulnerability. **Reference**
A link to additional information about the vulnerability from its source, such as the GitHub Advisory Database and the National Vulnerability Database (NVD), if available. ## Steps to set up PR comments ### Prerequisites Semgrep currently supports repositories hosted by Azure DevOps Cloud. In addition to finishing the previous steps in your deployment journey, it is recommended to have completed a **full scan** on your **default branch** for the repository in which you want to receive comments. ### Confirm your Semgrep account's connection PR comments are enabled by default for users who have connected their Azure DevOps project to Semgrep AppSec Platform. Confirm that you have the correct connection and access:
1. [Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Code](#configure-comments-for-semgrep-code). | | Software composition analysis (SCA) | Semgrep Supply Chain (SSC) | A comment appears based on the conditions you explicitly set in a [Supply Chain policy](/semgrep-supply-chain/policies) or when Semgrep detects a [license violation](/semgrep-supply-chain/license-compliance). | To receive Supply Chain comments, complete the steps in [Confirm account connection and access](#confirm-your-semgrep-accounts-connection) and [set up a policy](/semgrep-supply-chain/policies).
To receive license violation comments, [enable dependency search](/semgrep-supply-chain/dependency-search#enable-and-use-dependency-search). | | Secrets | Semgrep Secrets | A comment appears when a finding is generated by a rule in **Comment or Block mode**. A comment also appears for invalid findings and validation errors if these conditions are set to **Comment or Block mode**. | Complete the steps in the following sections:
1. [Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Secrets](#configure-comments-for-semgrep-secrets). | Comments from Supply Chain scans include the following information: **Risk**
A description of the vulnerability, including the types of attack it is vulnerable to. **Fix**
Indicates what versions to upgrade to, if any, that resolves or eliminates the vulnerability. **Reference**
A link to additional information about the vulnerability from its source, such as the [GitHub Advisory Database](https://github.com/advisories) and the [National Vulnerability Database (NVD)](https://nvd.nist.gov/vuln), if available. ## Supported Bitbucket plans * Any of the following Bitbucket plans are supported: * Cloud Free * Standard * Premium There are two ways in which you can integrate Semgrep comments into Bitbucket depending on the Bitbucket plan you use: * **Workspace access token**: If you use the Bitbucket Cloud Premium plan, you can create a workspace access token. This option saves time because you can create one access token for all repositories in the workspace. With one workspace access token, you can bulk-onboard more repositories at once from a whole workspace. However, you can also use the option of a repository access token to onboard repositories one by one. * **Repository access token**: If you do **not** have the Bitbucket Cloud Premium plan, create a separate repository access token for each repository where you want to use Semgrep. This configuration option is also useful if you have the Bitbucket Cloud Premium plan, but prefer to onboard repositories one by one instead of bulk onboarding.
1. [Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Code](#configure-comments-for-semgrep-code). | | Software composition analysis (SCA) | Semgrep Supply Chain (SSC) | A comment appears based on the conditions you explicitly set in a [Supply Chain policy](/semgrep-supply-chain/policies) or when Semgrep detects a [license violation](/semgrep-supply-chain/license-compliance). | To receive Supply Chain comments, complete the steps in [Confirm account connection and access](#confirm-your-semgrep-accounts-connection) and [set up a policy](/semgrep-supply-chain/policies).
To receive license violation comments, [enable dependency search](/semgrep-supply-chain/dependency-search#enable-and-use-dependency-search). | | Secrets | Semgrep Secrets | A comment appears when a finding is generated by a rule in **Comment or Block mode**. A comment also appears for invalid findings and validation errors if these conditions are set to **Comment or Block mode**. | Complete the steps in the following sections:
1. [Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Secrets](#configure-comments-for-semgrep-secrets). | Comments from Supply Chain scans include the following information: **Risk**
A description of the vulnerability, including the types of attack it is vulnerable to. **Fix**
Indicates what versions to upgrade to, if any, that resolves or eliminates the vulnerability. **Reference**
A link to additional information about the vulnerability from its source, such as the [GitHub Advisory Database](https://github.com/advisories) and the [National Vulnerability Database (NVD)](https://nvd.nist.gov/vuln), if available. ## Enable PR comments in Bitbucket ### Prerequisites * You must have a Bitbucket Data Center HTTP access token. Ensure that the [HTTP access token that you create](https://confluence.atlassian.com/bitbucketserver/http-access-tokens-939515499.html) has been granted **Project write** permissions. You'll provide this token to your CI provider during the setup process. * Semgrep has been tested with Bitbucket Data Center v8.19. If you are using a different version of BBDC and there are issues, please [reach out to support](/support). ### Confirm your Semgrep account's connection Confirm that you have the correct connection and access:
- How is my security posture doing over time?
- Is my backlog decreasing or increasing?
- Is the team addressing findings faster than new findings are coming in? | | [Secure guardrails](/secure-guardrails/secure-guardrails-in-semgrep) | Displays data relevant to the deployment and adoption of secure guardrails. It helps address the following:
- How many vulnerabilities did Semgrep prevent from entering production over time?
- Am I effectively introducing guardrails to my developers?
- Of the issues shown to developers, are they being fixed, or are they being ignored? | | Most findings by project | Lists projects arranged by most open findings to least, grouped by **product** or **severity**. Helps answer the following:
- Which of my projects have the most findings in a particular product area?
- Which of my projects have the most findings for a particular severity? | | Median open age | A graph showing the middle age of all Open findings, grouped by product or severity. Half of the open findings are older than this age, and half are newer. Helps you answer:
- What is the amount of time a finding remains open, by product or by severity? |
Hover over the chart to see a breakdown of findings by product for the selected time period. | | Backlog activity | Displays the number of new, net new, fixed, and ignored, including provisionally ignored, findings. A greater **Fixed** value is better.
Hover over the chart to see a breakdown of findings by triage state for that selected time period. | ## Secure guardrails This provides an overview of how secure guardrails in **PR or MR comments** are used in your organization, including how often Semgrep shows findings to developers, how the developers handle the findings, and how often Semgrep flags a finding as provisionally ignored. Other guardrail interfaces, such as the IDE or `pre-commit`, are not counted in this section. ### Key metrics | Key metrics | Description | | :---------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Findings shown to devs | Number of findings shown to developers in PR or MR comments (the numerator) against the total findings count (denominator). An upward or stable trend is better. | | Findings fixed in development | Number of findings that were fixed before they could be detected in a default branch or production backlog (numerator) against the total findings count in the specified time period (denominator). An upward or stable trend is better.
Hover over the chart to see a breakdown of findings by triage state for that selected time period. | ### Charts | Chart | Description | | :------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Secure guardrails adoption | Percent of new findings shown to developers over the specified time period. An upward or stable trend is better. | | Guardrails activity | This chart displays a breakdown of the status of findings shown to developers; whether they were ignored, provisionally ignored, fixed, or remained open. A greater **Fixed** value is better.
Hover over the chart to see a breakdown of findings by triage state for that selected time period. | ## Most findings by project A table listing projects from most open findings to least, grouped by product or severity. Lower values are better.
This appears if a finding fails the CI job. Organizations typically block PRs or MRs with failed jobs. **B - Finding description**
A human-written description always appears in a PR or MR comment, describing why your code is flagged. **References** may also be included to help you learn more about the finding. **C - Dataflow graph**
Some Code findings have a dataflow graph, which indicates that the finding was detected through %%taint analysis|taint\_analysis%%. The dataflow graph provides the lines of code identifying sources, sinks, and traces of unsanitized data flowing through your program. You can click the links on the boxes to take you to the lines of code. **D - Resolution or remediation section**
Various options are provided to help your resolve the finding. Depending on the type of finding, resolution options may vary. **E - Ignore instructions**
Click to view instructions about how to ignore the finding by replying to the comment. Depending on the features you have enabled, your PR comment can also appear more straightforward:
1. [Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Code](#configure-comments-for-semgrep-code). | | Software composition analysis (SCA) | Semgrep Supply Chain (SSC) | A comment appears based on the conditions you explicitly set in a [Supply Chain policy](/semgrep-supply-chain/policies) or when Semgrep detects a [license violation](/semgrep-supply-chain/license-compliance). | To receive Supply Chain comments, complete the steps in [Confirm account connection and access](#confirm-your-semgrep-accounts-connection) and [set up a policy](/semgrep-supply-chain/policies).
To receive license violation comments, [enable dependency search](/semgrep-supply-chain/dependency-search#enable-and-use-dependency-search). | | Secrets | Semgrep Secrets | A comment appears when a finding is generated by a rule in **Comment or Block mode**. A comment also appears for invalid findings and validation errors if these conditions are set to **Comment or Block mode**. | Complete the steps in the following sections:
1.[Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Secrets](#configure-comments-for-semgrep-secrets). | Comments from Supply Chain scans include the following information: **Risk**
A description of the vulnerability, including the types of attack it is vulnerable to. **Fix**
Indicates what versions to upgrade to, if any, that resolves or eliminates the vulnerability. **Reference**
A link to additional information about the vulnerability from its source, such as the [GitHub Advisory Database](https://github.com/advisories) and the [National Vulnerability Database (NVD)](https://nvd.nist.gov/vuln), if available. ## Steps to set up PR comments ### Prerequisites In addition to finishing the previous steps in your deployment journey, it is recommended to have completed a **full scan** on your **default branch** for the repository in which you want to receive comments. ### Confirm your Semgrep account's connection Confirm that you have the correct connection and access:
1. [Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Code](#configure-comments-for-semgrep-code). | | Software composition analysis (SCA) | Semgrep Supply Chain (SSC) | A comment appears based on the conditions you explicitly set in a [Supply Chain policy](/semgrep-supply-chain/policies) or when Semgrep detects a [license violation](/semgrep-supply-chain/license-compliance). | To receive Supply Chain comments, complete the steps in [Confirm account connection and access](#confirm-your-semgrep-accounts-connection) and [set up a policy](/semgrep-supply-chain/policies).
To receive license violation comments, [enable dependency search](/semgrep-supply-chain/dependency-search#enable-and-use-dependency-search). | | Secrets | Semgrep Secrets | A comment appears when a finding is generated by a rule in **Comment or Block mode**. A comment also appears for invalid findings and validation errors if these conditions are set to **Comment or Block mode**. | Complete the steps in the following sections:
1. [Confirm your Semgrep account's connection and access to your source code manager](#confirm-your-semgrep-accounts-connection).
2. [Configure comments for Semgrep Secrets](#configure-comments-for-semgrep-secrets). | Comments from Supply Chain scans include the following information: **Risk**
A description of the vulnerability, including the types of attack it is vulnerable to. **Fix**
Indicates what versions to upgrade to, if any, that resolves or eliminates the vulnerability. **Reference**
A link to additional information about the vulnerability from its source, such as the [GitHub Advisory Database](https://github.com/advisories) and the [National Vulnerability Database (NVD)](https://nvd.nist.gov/vuln), if available. ## Steps to set up MR comments ### Confirm your Semgrep account's connection PR comments are enabled by default for users who have connected their GitLab group to Semgrep AppSec Platform. Confirm that you have the correct connection and access:
β’ Semgrep AppSec Platform
β’ For Semgrep Code and Supply Chain: User-defined notifications
Set rules to this mode to evaluate their true positive rate and other criteria you may have. By keeping rules in Monitor, developers do not receive potentially noisy findings in their PRs or MRs. | | Comment | Rules in **Comment mode** display findings in:
β’ Developers' PRs or MRs
β’ Semgrep AppSec Platform
β’ **For Semgrep Code and Supply Chain:** User-defined notifications
Set rules that have met your performance criteria to this mode when you are ready to display findings to developers. | | Block | Rules in **Block mode** cause the scan job to fail with an exit code of `1` if Semgrep Secrets detects a finding from these rules. You can use this result to enforce a block on the PR or MR. For example, GitHub users can enable branch protection and set the PR to fail if the Semgrep step fails.
These rules display findings in:
β’ Developers' PRs or MRs
β’ Semgrep AppSec Platform
β’ **For Semgrep Code and Supply Chain:** User-defined notifications
These are typically high-confidence, high-severity rules. |
`pullrequest:read` | `repository:write`
`pullrequest:write` | | Bitbucket Data Center | `repository:read` | `repository:write` | | GitHub.com and GitHub Enterprise Server | `contents:read` | `contents:write` | | GitLab and GitLab Self-Managed | `read_repository` | `write_repository` |
Grant code access to Semgrep with a private GitHub app
If you are an **existing** Semgrep user and your private Semgrep GitHub app does not have code access enabled, follow these steps to update the app and grant code access to Semgrep. GitHub grants code access through **Repository permissions > Contents** under **Developer Settings**. The installation page at **Settings > Applications** might list read/write permissions but those do not enable file access.[https://dev.azure.com/ORGANIZATION\_NAME](https://dev.azure.com/ORGANIZATION_NAME)
[https://dev.azure.com/ORGANIZATION\_NAME/\_usersSettings/tokens](https://dev.azure.com/ORGANIZATION_NAME/_usersSettings/tokens).
[https://bitbucket.org/WORKSPACE/workspace/settings/access-keys](https://bitbucket.org/WORKSPACE/workspace/settings/access-keys).
[https://github.com/settings/personal-access-tokens](https://github.com/settings/personal-access-tokens). GitHub Enterprise Server users must replace the [https://github.com](https://github.com) base URL with the base URL of their GitHub Enterprise Server instance.
[https://gitlab.com/groups/GROUP/-/settings/access\_tokens](https://gitlab.com/groups/GROUP/-/settings/access_tokens).
GitLab Self-Managed users must replace the [https://gitlab.com](https://gitlab.com) base URL with the base URL of the GitLab Self-Managed instance.
/semgrep\_subscribe ACCOUNT\_NAME/REPOSITORY\_NAME
@Semgrep in conversations that the app is in. | Enables the Semgrep Slack app to respond when users mention it in the chat. |
| **[channels:read](https://api.slack.com/scopes/channels:read)** | View basic information about public channels in a workspace. | Basic channel information such as channel\_id is used to ensure that Semgrep findings (results) are sent to the appropriate channel. |
| **[chat:write](https://api.slack.com/scopes/chat:write)** | Send messages as @Semgrep. | Enables the Semgrep Slack app to send findings to channels. |
| **[chat:write.customize](https://api.slack.com/scopes/chat:write.customize)** | Send messages as @Semgrep with a customized username and avatar. | Helps users identify Semgrep Slack app messages through the use of an image and username. |
| **[chat:write.public](https://api.slack.com/scopes/chat:write.public)** | Send messages to channels @Semgrep isn't a member of. | Enables users to invoke Semgrep Slack app features in any public channel using the slash command. |
| **[commands](https://api.slack.com/scopes/commands)** | Add shortcuts or slash commands that people can use. | Enables the Semgrep Slack app to register custom slash commands such as /semgrep\_subscribe used for notification subscription. |
| **[emoji:read](https://api.slack.com/scopes/emoji:read)** | View custom emoji in a workspace. | Allows Semgrep to support a workspace's custom emojis. |
| **[im:write](https://api.slack.com/scopes/im:write)** | Start direct messages with people. | Allows users to interact with the Semgrep Slack app and use the slash commands in direct messages. |
| **[links:write](https://api.slack.com/scopes/links:write)** | Show previews of URLs in messages. | Enables Semgrep Slack app to include links in messages. |
| **[users:read](https://api.slack.com/scopes/users:read)** | View profile details about people in a workspace. | Enables Semgrep Slack app to correctly address users in messages. |
| **[users:write](https://api.slack.com/scopes/users:write)** | Set presence for Semgrep. | Used by the Semgrep Slack app to interact with the workspace and enables users to add the Semgrep Slack app to relevant channels. |
| **[workflow.steps:execute](https://api.slack.com/scopes/workflow.steps:execute)** | Add steps that people can use in Workflow Builder. | Enables Semgrep to make use of modals and drop-down boxes when a user creates or updates their notifications. |
| **[groups:read](https://api.slack.com/scopes/groups:read)** | View basic information about private channels that your Slack app has been added to. | Semgrep Slack app uses channels\_id\_changed to update its notifications configuration if the channel that receives findings is updated. This ensures that you are able to receive findings ever renaming a channel. |
| **[team:read](https://api.slack.com/scopes/team:read)** | View the name, email domain, and icon for workspaces your slack app is connected to. | Semgrep Slack app uses team\_name\_changed to update its notifications configuration if the team name is updated. This ensures that you are able to receive findings notifications even after renaming your team. |
| **[channels:read](https://api.slack.com/scopes/channels:read)** | View basic information about public channels in a workspace. | Enables Semgrep Slack app to monitor if channels that receive Semgrep findings have been deleted or archived. |
#### Additional resources
* [https://api.slack.com/apps](https://api.slack.com/apps)
* [https://api.slack.com/messaging/webhooks#enable\_webhooks](https://api.slack.com/messaging/webhooks#enable_webhooks)
# View runtime context from Sysdig
Source: https://docs.semgrep.dev/semgrep-appsec-platform/sysdig
The Semgrep Sysdig integration can ingest runtime context from your Sysdig account into Semgrep AppSec Platform. This allows you to prioritize findings based on deployment status.
## Prerequisites
Before proceeding, ensure that:
* You have a license for **Semgrep Supply Chain**
* You have the following tools and integrations set up in your Sysdig account:
* [i. Select the rules by clicking the checkboxes next to their names, then click **Change scanning behavior (n)**. If you're modifying only one rule, you can click the ruleβs link in the **Projects scanning** column.
ii. The **Projects scanning** dialog appears. You can choose to use the rules with **All projects**, **Selected projects** by **Project name**, **Selected projects** by **Tags**, **All with exceptions**, or **None (disable)**.
iii. To use the rules with all of your projects, click **All projects**, then click **Save**.
iv. To use the rules with some of your projects, click **Selected projects**, then click the checkboxes next to the projects to which the rule applies. Click **Save** to proceed.
v. To exclude specific projects, click **All with exceptions**, then select the projects to which the rule doesnβt apply. Click **Save** to proceed.
vi. To prevent the rules from being used at all, click **None (Disable)**, then **Save** to proceed.
i. Click **+ Create automation**.
ii. Provide a **Policy name**.
iii. Set the **Scope** of the policy by selecting all of the projects to which this policy applies.
iv. Define the **Conditions** that trigger the policy by clicking **+ Add condition** to expand a drop-down list of attributes that you can use. Select the attribute, then set the specific conditions. * For example, you can select **Severity**, then complete the conditional statement provided to read **When Severity is any of Critical, High**. * You can define as many conditions as necessary, and Semgrep treats them additively. v. Choose the **Actions** that occur if there are findings that trigger the policy by clicking **+ Add action**. You can choose multiple **Actions**, including:
Scope: All projects | | :--------- | :--------------------------------- | | **Rule B** | Enabled.
Scope: All projects | | **Rule C** | Enabled.
Scope: All projects | | **Rule D** | Enabled.
Scope: All projects | ### Remediation policy | | | | | :--------------- | :---------------- | :------------------------------------------------------------ | | **Automation 1** | Name | Comment on PR or MR with findings | | **Automation 1** | Scope | All projects | | **Automation 1** | Conditions | Rule is one of:
- Rule B
- Rule C
- Rule D
A `semgrep_scan` object contains information about the CI job and other scan parameters, such as ignored files. Semgrep sends a single `semgrep_scan` object **every time a scan is run**. This includes diff-aware scans, full scans, and scans that have no findings. `semgrep_finding` JSON object
A `semgrep_finding` object is a single record of a new finding. Semgrep sends new `semgrep_finding` objects based on how you have configured your notifications in Policies. See [Set up webhooks](#set-up-webhooks) to learn more. ## Set up webhooks Perform these steps in Semgrep AppSec Platform to set up webhooks:
ββii. Authentication URL You can find both values at a later date under [tenant info](https://app.wiz.io/tenant-info/general).
β’ Cross-file dataflow analysis
β’ 150+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | C# | \*\*Generally available \*\*
β’ Cross-file dataflow analysis
β’ Supports up to C# 13
β’ 170+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules
β’ Supports up to C# 7.0 | | Go | **Generally available**
β’ Cross-file dataflow analysis
β’ 80+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | Java | **Generally available**
β’ Cross-file dataflow analysis
β’ Framework-specific control flow analysis
β’ 190+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | JavaScript | **Generally available**
β’ Cross-file dataflow analysis
β’ Framework-specific control flow analysis
β’ 250+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | Kotlin | \*\*Generally available \*\*
β’ Cross-file dataflow analysis
β’ 60+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | [Python](/languages/python) | **Generally available**
β’ Cross-file dataflow analysis
β’ Framework-specific control flow analysis
β’ 710+ Pro rules
β’ See [Python-specific support details](/languages/python) | Community supported
β’ Limited to single-function analysis
β’ Community rules | | Typescript | \*\*Generally available \*\*
β’ Cross-file dataflow analysis
β’ Framework-specific control flow analysis
β’ 230+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | Ruby | \*\*Generally available \*\*
β’ Cross-function dataflow analysis
β’ 40+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | Rust | \*\*Generally available \*\*
β’ Cross-function dataflow analysis
β’ 40+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | JSX | \*\*Generally available \*\*
β’ Cross-function dataflow analysis
β’ 70+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | PHP | \*\*Generally available \*\*
β’ Cross-function dataflow analysis
β’ 50+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | Scala | \*\*Generally available \*\*
β’ Cross-function dataflow analysis
β’ Community rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | Swift | \*\*Generally available \*\*
β’ Cross-function dataflow analysis
β’ 60+ Pro rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | Terraform | **Generally available**
β’ Cross-function dataflow analysis
β’ Community rules | Community supported
β’ Limited to single-function analysis
β’ Community rules | | Generic | \*\*Generally available \*\* | Community supported | | JSON | \*\*Generally available \*\* | Community supported | | APEX | **Beta** | Not available | | Elixir | **Beta** | Not available |
semgrep ci \|\| true | CI **passes** on blocking findings and internal errors. |
To change Semgrep's behavior, modify your pipeline or job file, specifically the `semgrep ci` command, to the CI option that best fits your needs. For example, GitHub users should edit the `semgrep.yml` workflow file and include the following under the `run` key:
```yaml theme={null}
run:
semgrep ci --suppress-errors
```
GitLab users would include the following under the `script` key:
```yaml theme={null}
script:
semgrep ci --suppress-errors
```
If you use any other CI provider, refer to its documentation for information on where to provide this information. Additionally, see the sample configurations in the following section.
## Sample configurations for blocking findings and errors
The following is a sample `.semgrep.yml` file you can use with GitHub Actions. Semgrep's default behavior regarding blocking findings and errors applies here:
* Semgrep returns exit code `1` if there are blocking findings
* Semgrep returns exit code `0` if there are *no* blocking findings, even if there are internal errors. Semgrep does, however, send an anonymous report to the crash-reporting server.
This means that, by default, Semgrep doesn't report statuses other than `0` or `1`.
```yaml expandable theme={null}
# Name of this GitHub Actions workflow.
name: Semgrep
on:
# Scan changed files in PRs (diff-aware scanning):
pull_request: {}
# Scan on-demand through GitHub Actions interface:
workflow_dispatch: {}
# Scan mainline branches if there are changes to .github/workflows/semgrep.yml:
push:
branches:
- main
- master
paths:
- .github/workflows/semgrep.yml
# Schedule the CI job (this method uses cron syntax):
schedule:
- cron: '20 17 * * *' # Sets Semgrep to scan every day at 17:20 UTC.
# It is recommended to change the schedule to a random time.
permissions:
contents: read
jobs:
semgrep:
# User definable name of this GitHub Actions job.
name: semgrep/ci
# If you are self-hosting, change the following `runs-on` value:
runs-on: ubuntu-latest
container:
# A Docker image with Semgrep installed. Do not change this.
image: semgrep/semgrep
# Skip any PR created by dependabot to avoid permission issues:
if: (github.actor != 'dependabot[bot]')
steps:
# Fetch project source with GitHub Actions Checkout. Use either v3 or v4.
- uses: actions/checkout@v6
# Run the "semgrep ci" command on the command line of the docker image.
- run: semgrep ci
env:
# Connect to Semgrep AppSec Platform through your SEMGREP_APP_TOKEN.
# Generate a token from Semgrep AppSec Platform > Settings
# and add it to your GitHub secrets.
SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}
```
Optionally, you can explicitly indicate that Semgrep is using the default settings by including the `--suppress-errors` flag. The modified portion of the configuration file is as follows:
```yaml theme={null}
steps:
- uses: actions/checkout@v6
- name: Scan and suppress internal errors
run: semgrep ci --suppress-errors
```
The following code snippets display the position of the default flag in the configuration files of various CI providers:
**Note**: [AI-powered detection](/semgrep-code/ai-powered-detection-concepts) findings remain **Open** even when Multimodal flags them as false positives. They are **not** automatically set to **Provisionally ignored**; change the status to **Ignored** if you want to dismiss them. | | **To fix** | Findings that you have decided to fix. Commonly used to indicate that these findings are tracked in Jira or assigned to developers for further work. | | **Fixed** | Fixed findings were detected in a previous scan but are no longer detected in the most recent scan of that same branch due to changes in the code. | | **Ignored** | Findings marked as ignored are present in the code but have been labeled unimportant. Ignore false positives or deprioritized issues. Mark findings as [ignored through Semgrep AppSec Platform](/semgrep-code/triage-remediation) or by adding a [nosemgrep code comment](/ignoring-files-folders-code/#reference-summary). You can also provide a reason for ignoring a finding: **False positive**, **Acceptable risk**, **No time to fix**. | | **Closed** | Vulnerabilities that are no longer detected after a scan. This can be due to changes in the underlying rule or the code. | ### Removed findings Findings can also be **removed**. Semgrep considers a finding removed if it is not found in the most recent scan of the branch where Semgrep initially detected it due to any of the following conditions: * The rule that detected the finding isn't enabled in the policy anymore. * The rule that detected the finding was updated in a way that it no longer detects the finding. * The file path where the finding appeared is no longer found. The file path was deleted, renamed, added to a `.semgrepignore` file, added to a `.gitignore` file, or added to the list of ignored paths in Semgrep AppSec Platform. * For GitHub organization accounts: the pull request or merge request where the finding was detected has been closed without merging. Your removed findings do not count toward the fix rate or the number of findings. The removed findings also do not appear in Semgrep AppSec Platform. ### Triage behavior across refs and branches * When you triage a finding as ignored, reviewing, fixing, or reopened, Semgrep always triages across other branches and [Git references](https://git-scm.com/book/en/v2/Git-Internals-Git-References) (refs). * At scan time, there's automatic triaging that occurs in specific cases, and the behavior changes depending on the type of scan: * **Full scans**: if the current branch includes a finding that was * Previously introduced in another branch ***and*** * Triaged to a specific state
**Then** the finding in the current branch is triaged to that same state. * **Diff-aware scan**: findings introduced in a diff-aware scan are **not** automatically triaged at scan time, even if there are other instances of that finding on branches that have been triaged. # Findings in CI Source: https://docs.semgrep.dev/semgrep-ci/findings-ci-1 When running any Semgrep product in CI, Semgrep is able to track the lifetime of an individual finding. When configured to perform a diff-aware scan, Semgrep only shows new findings relative to some specified baseline commit. In the code, a Semgrep finding in CI is defined by a 4-tuple: ```text theme={null} (rule ID, file path, syntactic context, index) ``` These states correspond to: 1. `rule ID`: The rule's ID within the Semgrep ecosystem. 2. `file path`: The filesystem path where the finding occurred. 3. `syntactic context`: The lines of code corresponding to the finding. 4. `index`: An index into identical findings within a file. This is used to disambiguate findings if the same `syntactic context` occurs multiple times in the same file. ## Semgrep Code findings Semgrep AppSec Platform builds on CI findings to track status and provide additional context for managing findings within your organization. A finding can be one of four statuses in Semgrep AppSec Platform: * `OPEN` * `PROVISIONALLY_IGNORED` * `REVIEWING` * `FIXING` * `IGNORED` * `FIXED` ### Finding status You can manage finding status through triage in Semgrep AppSec Platform's **Findings** page. The finding statuses are as follows: | Status | Description | | :------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Open** | Findings are open by default. A finding is open if it was present the last time Semgrep scanned the code and has not been ignored. An open finding represents a match between the code and a rule enabled in the repository. Open findings require action, such as rewriting the code to eliminate the detected vulnerability. | | **Reviewing** | Indicates that the finding requires investigation to determine what the next steps in the triage process should be. | | **Provisionally ignored** | Rule-based Semgrep Code findings that Semgrep Multimodal's [Autotriage](/semgrep-multimodal/overview#autotriage) has flagged as likely false positives. You can change the status to **Ignored** if you agree with Multimodal's assessment. Otherwise, you can change the status to **To fix** if you disagree.
**Note**: [AI-powered detection](/semgrep-code/ai-powered-detection-concepts) findings remain **Open** even when Multimodal flags them as false positives. They are **not** automatically set to **Provisionally ignored**; change the status to **Ignored** if you want to dismiss them. | | **To fix** | Findings that you have decided to fix. Commonly used to indicate that these findings are tracked in Jira or assigned to developers for further work. | | **Fixed** | Fixed findings were detected in a previous scan but are no longer detected in the most recent scan of that same branch due to changes in the code. | | **Ignored** | Findings marked as ignored are present in the code but have been labeled unimportant. Ignore false positives or deprioritized issues. Mark findings as [ignored through Semgrep AppSec Platform](/semgrep-code/triage-remediation) or by adding a [nosemgrep code comment](/ignoring-files-folders-code/#reference-summary). You can also provide a reason for ignoring a finding: **False positive**, **Acceptable risk**, **No time to fix**. | | **Closed** | Vulnerabilities that are no longer detected after a scan. This can be due to changes in the underlying rule or the code. | ### Removed findings Findings can also be **removed**. Semgrep considers a finding removed if it is not found in the most recent scan of the branch where Semgrep initially detected it due to any of the following conditions: * The rule that detected the finding isn't enabled in the policy anymore. * The rule that detected the finding was updated in a way that it no longer detects the finding. * The file path where the finding appeared is no longer found. The file path was deleted, renamed, added to a `.semgrepignore` file, added to a `.gitignore` file, or added to the list of ignored paths in Semgrep AppSec Platform. * For GitHub organization accounts: the pull request or merge request where the finding was detected has been closed without merging. Your removed findings do not count toward the fix rate or the number of findings. The removed findings also do not appear in Semgrep AppSec Platform. ### Triage behavior across refs and branches * When you triage a finding as ignored, reviewing, fixing, or reopened, Semgrep always triages across other branches and [Git references](https://git-scm.com/book/en/v2/Git-Internals-Git-References) (refs). * At scan time, there's automatic triaging that occurs in specific cases, and the behavior changes depending on the type of scan: * **Full scans**: if the current branch includes a finding that was * Previously introduced in another branch ***and*** * Triaged to a specific state
**Then** the finding in the current branch is triaged to that same state. * **Diff-aware scan**: findings introduced in a diff-aware scan are **not** automatically triaged at scan time, even if there are other instances of that finding on branches that have been triaged. # Set up Semgrep Network Broker Source: https://docs.semgrep.dev/semgrep-ci/network-broker Semgrep Network Broker facilitates secure access between Semgrep and your private network. The Network Broker creates a WireGuard VPN tunnel to the Semgrep backend and proxies **inbound** HTTP requests from Semgrep to the customer through the tunnel. This allows Semgrep to communicate with private network resources like a Source Code Manager (SCM) without exposing them to the public internet.
/REPOSITORY-ROOT-DIRECTORY/.github/workflows/semgrep.yml.
/REPOSITORY-ROOT-DIRECTORY/.github/workflows/semgrep.yml.
View and open rules owned by your organization or available through the [Semgrep Registry](https://semgrep.dev/r). **Rule editor**
Enter your rule's YAML in this pane. This pane supports both structure and advanced modes. This pane also contains metadata editing functionality in Structure mode, and match review functionality in Advanced mode. **Sample code**
Enter test code in this pane and click **Run** to verify that the rule performs as intended. A matches panel appears after Semgrep runs to display matches and tests. **Top menu**
Save, share, and add your rule to one of your policies. ### Group Registry rules By default, Semgrep Registry rules are grouped by **directory**. Most of these directories correspond to languages. The Library can also be grouped by **rulesets**, which are rules sorted by category, such as security, best practices, and frameworks. To group by ruleset, right-click on the empty space on the registry's name entry and select **Group by ruleset**. ## Create a rule To create a rule, click **Create rule** on the splash page or the **(+) sign** next to the Library label. Semgrep Editor offers two rule-writing modes: **Structure mode (beta)**
Structure mode is a hybrid interface that offers guidance for rule writing while supporting additional features the way advanced mode does. **Advanced mode**
Advanced mode provides the minimum required YAML keys for a Semgrep rule. To complete the rule, you must fill in additional keys, such as pattern operators or metadata. ### Write a rule using structure mode (beta) Structure mode is a UI-based ruled writing editor that guides you through the process of writing a rule. Structure mode features include: * **Match badges**: Match badges are visual indicators paired with pattern operators. The match badge shows the number of matches associated with each pattern operator. * **Automatic indentation**: When adding a new pattern to a nested operator such as `patterns` or `pattern-either`, the editor automatically indents sub-patterns correctly. * **Differentiation between patterns and pattern constraints**: A pattern is one of six different operators that describes zero or more locations in a rule. These include `pattern`, `any`, `all`, `inside`, `regex`, and `not`. You can combine these in prescribed ways, such as `any` and `all`, using range union and intersection, but they still define ranges. Pattern constraints describe Boolean constrains that must be met for a match to occur. If the constraint doesn't hold, then the ranges determined by the pattern operators aren't applicable. * **Interoperability with advanced mode**: You can write a rule using structure mode and view or export it in YAML, or you can paste in the YAML for a rule and edit it with structure mode. * **Drag and drop**" You can move around the elements of a rule using drag and drop. * **Pattern disabling**: You can toggle individual patterns on or off for actions like testing. To write a **search** rule using structure mode:
- Low
- Medium
- High
- Critical
Only Semgrep Supply Chain and Code findings provide this field. | | Category | The finding's category, such as **best practices**, **security**, or **correctness**. | | Is pro rule | Boolean value that returns `TRUE` if the rule that generated the finding is a pro rule. | | Assistant triage result | Provides Semgrep Multimodal's assessment. Possible values are `True positive` or `False positive`. These values appear only if Multimodal is enabled. | | Assistant triage reason | A short AI-generated reason why Multimodal thinks the finding is a true or false positive. These values appear only if Multimodal is enabled. | The following fields are exclusive to **Supply Chain** scans: | Field | Description | | :----------- | :--------------------------------------------------------------------------------------------------------------- | | Dependency | The name of the dependency where the findings was found. | | Reachability | The reachability status of the finding, such as **Reachable**, **No Reachability Analysis**, or **Unreachable**. | | Transitivity | States whether the finding originates from a direct or transitive dependency. | | CVE | The CVE number that the finding is assigned to. | | EPSS | The EPSS score, which estimates the likelihood that a software vulnerability can be exploited in the wild. | The following fields are exclusive to **Secrets** scans: | Field | Description | | :----------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Secret type | Possible values include **AI-detected**, **Generic secret**, **Connection URI**, and so on. | | Validation | States whether or not the secret was validated. | | Project visibility | States whether the project (repository) is public or private. This feature supports GitHub-hosted repositories only. It returns an **Unknown** value for non-GitHub SCMs. |
The top header consists of: * **Policies view drop-down**, which lets you choose between: * Grouping rules by vulnerability class * No grouping *
Displays filters to quickly select and perform operations on rules in bulk. See [Policies filter reference](#policies-filter-reference) for more information. **Rule pane**
The rule pane displays the rules that Semgrep scans use to detect findings and allows you to edit their assigned rule modes. You can make these edits either one by one or through bulk editing of many rules. You can also use the **Search for rule names or ids** box. See [Policies filter reference](#policies-filter-reference) for more information. ### Policies page filters This section defines the Policies page filters: | Filter | Description | Examples or possible values | | :--------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Modes | Filter by the workflow action Semgrep performs when a rule detects a finding. An additional filter, **Disabled**, is provided for rules that you have turned off and are no longer included for scanning. | See [Rule modes](#block-a-pr-or-mr-through-rule-modes) documentation. | | Category | Filter by the type of security issue or vulnerability that the rule detects. |
- Dangerous method or function
- SQL injection
- Active debug code
- Critical
- High
- Medium
- Low
- High
- Medium
- Low
- **Pro:** Authored by Semgrep with cross-file (interfile) and cross-function (interprocedural) analysis capabilities, providing you with enhanced scan accuracy. For more information, see
Pro rules. - **Community:** Authored by Semgrep, Inc or external contributors such as Trail of Bits.
- **Custom:** Rules created within your Semgrep organization. For more information, see
Private rules.
- Python
- JavaScript
- Ruby
- 10
- 100
- 500
- Security
- Code injection
- PHP
- High
- Medium
- Low
- High
- Medium
- Low
- **Pro:** Authored by Semgrep with cross-file (interfile) and cross-function (interprocedural) analysis capabilities, providing you with enhanced scan accuracy. For more information, see
Pro rules. - **Community:** Authored by Semgrep, Inc or external contributors such as Trail of Bits.
- **Custom:** Rules created within your Semgrep organization. For more information, see
Private rules.
- Semgrep AppSec Platform
- **For Semgrep Code and Supply Chain**: User-defined notifications
- Developers' PRs or MRs
- Semgrep AppSec Platform
- **For Semgrep Code and Supply Chain**: User-defined notifications
These rules display findings in:
- Developers' PRs or MRs
- Semgrep AppSec Platform
- **For Semgrep Code and Supply Chain**: User-defined notifications
Scans the repository in its entirety. It is recommended to perform full scans on mainline branches, such as `master` or `main`. Full scans are typically performed on a scheduled basis or on merge to a default branch. **Diff-aware scan**
Diff-aware scans are performed on non-mainline branches, such as in pull requests and merge requests. Diff-aware scans traverse the repository's files based on the commit where the branch diverged from the mainline branch. ## How Semgrep distinguishes between new and duplicate findings Semgrep generates a finding whenever it scans a repository and one of its rules matches a piece of code. Since Semgrep usually scans a repository multiple times, it needs a way to track the same finding in a file over time. Semgrep does this using two types of fingerprints: `match_based_id` and `syntactic_id`.
**Note**: [AI-powered detection](/semgrep-code/ai-powered-detection-concepts) findings remain **Open** even when Multimodal flags them as false positives. They are **not** automatically set to **Provisionally ignored**; change the status to **Ignored** if you want to dismiss them. | | **To fix** | Findings that you have decided to fix. Commonly used to indicate that these findings are tracked in Jira or assigned to developers for further work. | | **Fixed** | Fixed findings were detected in a previous scan but are no longer detected in the most recent scan of that same branch due to changes in the code. | | **Ignored** | Findings marked as ignored are present in the code but have been labeled unimportant. Ignore false positives or deprioritized issues. Mark findings as [ignored through Semgrep AppSec Platform](/semgrep-code/triage-remediation) or by adding a [nosemgrep code comment](/ignoring-files-folders-code/#reference-summary). You can also provide a reason for ignoring a finding: **False positive**, **Acceptable risk**, **No time to fix**. | | **Closed** | Vulnerabilities that are no longer detected after a scan. This can be due to changes in the underlying rule or the code. | ### Removed findings Findings can also be **removed**. Semgrep considers a finding removed if it is not found in the most recent scan of the branch where Semgrep initially detected it due to any of the following conditions: * The rule that detected the finding is no longer enabled in the policy. * The rule that detected the finding was updated in a way that it no longer detects the finding. * The file path where the finding appeared is no longer found. The file path was deleted, renamed, added to a `.semgrepignore` file, added to a `.gitignore` file, or added to the list of ignored paths in Semgrep AppSec Platform. * For GitHub organization accounts: the pull request or merge request where the finding was detected has been closed without merging. Your removed findings do not count toward the fix rate or the number of findings. The removed findings also do not appear in Semgrep AppSec Platform. ### Triage behavior across refs and branches * When you triage a finding as ignored, reviewing, fixing, or reopened, Semgrep always triages across other branches and [Git references](https://git-scm.com/book/en/v2/Git-Internals-Git-References) (refs). * At scan time, there's automatic triaging that occurs in specific cases, and the behavior changes depending on the type of scan: * **Full scans**: if the current branch includes a finding that was * Previously introduced in another branch ***and*** * Triaged to a specific state\ **Then** the finding in the current branch is triaged to that same state. * **Diff-aware scan**: findings introduced in a diff-aware scan are **not** automatically triaged at scan time, even if there are other instances of that finding on branches that have been triaged. ## Triage and remediation The following sections show you how to manage your findings by: * Fixing the underlying code * Disabling a rule or a ruleset * Ignoring a finding * Reopening a finding Note that some actions, such as ignoring and reopening findings, require different steps based on whether you have chosen **Group by Rule** or **No Grouping** when viewing your results on the **Findings** page. ## Review provisionally ignored findings If you have Semgrep Multimodal enabled, review the findings that have been **provisionally ignored**. These are rule-based Semgrep Code findings that Semgrep Multimodal's [Autotriage](/semgrep-multimodal/overview#autotriage) has flagged as likely false positives. For each finding, you can change the status to **Ignored** if you agree with Multimodal's assessment. Otherwise, you can change the status to **To fix** if you disagree. Findings with a status of **provisionally ignored** block pull requests and merge requests if the matching rule is included in a blocking policy.
/fp \ | Triage a finding as **Ignored** with the triage reason **false positive**. Provide a \ with information about the triage decision. |
| /ar \ | Triage a finding as **Ignored** with the triage reason **acceptable risk**. Provide a \ with information about the triage decision. |
| /other \ | Triage a finding as **Ignored** without specifying the reason; the triage reason value is set to **No triage reason**. Provide a \ with information about the triage decision. |
| /open \ | Reopen a finding that has been triaged as **Ignored**. Optionally, provide a \ with information about the decision to reopen the finding. |
/semgrep ignore \ - triage a finding as **Ignored**.
* /semgrep open \ - reopen a finding that has been triaged as **Ignored**.
Includes an open source, lightweight SAST scanner and rules in the [Semgrep Registry](https://semgrep.dev/r/) with **open source licenses**. You can also write your own custom rules. Semgrep CE also includes the Visual Studio Code (VS Code) and IntelliJ extensions. The Community Edition is best for small teams or personal projects. **Semgrep AppSec Platform (Semgrep)**
Refers to a proprietary software suite tailored to support AppSec engineers through the entire software development life cycle (SDLC). Best for deploying security programs throughout their organization. Many of Semgrep's features support the deployment of [secure guardrails](/secure-guardrails/secure-guardrails-in-semgrep). Semgrep includes the following products: **Semgrep Code**
A SAST scanner that uses cross-file (interfile) and cross-function (intrafile) analysis for improved results over Semgrep Community Edition. Semgrep Code includes rules written by Semgrep's Security Research team, called **Pro Rules**. These rules use cross-file analysis to reduce false positives. **Semgrep Supply Chain**
A high-signal dependency scanner that detects reachable vulnerabilities in open source third-party libraries and functions across the software development life cycle (SDLC). **Semgrep Secrets**
A secrets scanner that, in addition to detecting secrets, validates these leaked secrets on a variety of services to help you prioritize active secrets.
###### Semgrep Community Edition (SAST) * [30+ Community supported languages](/semgrep-ce-languages#semgrep-code-and-community-edition) * [
Includes an open source, lightweight SAST scanner and rules in the [Semgrep Registry](https://semgrep.dev/r/) with **open source licenses**. You can also write your own custom rules. Semgrep CE also includes the Visual Studio Code (VS Code) and IntelliJ extensions. The Community Edition is best for small teams or personal projects. **Semgrep AppSec Platform (Semgrep)**
Refers to a proprietary software suite tailored to support AppSec engineers through the entire software development life cycle (SDLC). Best for deploying security programs throughout their organization. Many of Semgrep's features support the deployment of [secure guardrails](/secure-guardrails/secure-guardrails-in-semgrep). Semgrep includes the following products: **Semgrep Code**
A SAST scanner that uses cross-file (interfile) and cross-function (intrafile) analysis for improved results over Semgrep Community Edition. Semgrep Code includes rules written by Semgrep's Security Research team, called **Pro Rules**. These rules use cross-file analysis to reduce false positives. **Semgrep Supply Chain**
A high-signal dependency scanner that detects reachable vulnerabilities in open source third-party libraries and functions across the software development life cycle (SDLC). **Semgrep Secrets**
A secrets scanner that, in addition to detecting secrets, validates these leaked secrets on a variety of services to help you prioritize active secrets.
###### Semgrep Community Edition (SAST) * [30+ Community supported languages](/semgrep-ce-languages#semgrep-code-and-community-edition) * [
- Low
- Medium
- High
- Critical
- Scans from local developer machines.
- Scans from any non-GitHub source code manager, such as GitLab.
Only Semgrep Supply Chain and Code findings provide this field. | | Category | The finding's category, such as **best practices**, **security**, or **correctness**. | | Is pro rule | Boolean value that returns `TRUE` if the rule that generated the finding is a pro rule. | | Assistant triage result | Provides Semgrep Multimodal's assessment. Possible values are `True positive` or `False positive`. These values appear only if Multimodal is enabled. | | Assistant triage reason | A short AI-generated reason why Multimodal thinks the finding is a true or false positive. These values appear only if Multimodal is enabled. | The following fields are exclusive to **Supply Chain** scans: | Field | Description | | :----------- | :--------------------------------------------------------------------------------------------------------------- | | Dependency | The name of the dependency where the findings was found. | | Reachability | The reachability status of the finding, such as **Reachable**, **No Reachability Analysis**, or **Unreachable**. | | Transitivity | States whether the finding originates from a direct or transitive dependency. | | CVE | The CVE number that the finding is assigned to. | | EPSS | The EPSS score, which estimates the likelihood that a software vulnerability can be exploited in the wild. | The following fields are exclusive to **Secrets** scans: | Field | Description | | :----------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Secret type | Possible values include **AI-detected**, **Generic secret**, **Connection URI**, and so on. | | Validation | States whether or not the secret was validated. | | Project visibility | States whether the project (repository) is public or private. This feature supports GitHub-hosted repositories only. It returns an **Unknown** value for non-GitHub SCMs. |
- **Low**: low privilege; for example, write-only access like a webhook
- **Medium**: may have read and write access depending on what scope the account has
- **High** and **Critical**: has access to critical resources or full account access
- **Pro:** Authored by Semgrep.
- **Custom:** Rules created within your Semgrep organization.
- Semgrep AppSec Platform
- For Semgrep Code and Supply Chain: User-defined notifications
- Developers' PRs or MRs
- Semgrep AppSec Platform
- For Semgrep Code and Supply Chain: User-defined notifications
These rules display findings in:
- Developers' PRs or MRs
- Semgrep AppSec Platform
- For Semgrep Code and Supply Chain: User-defined notifications
- Critical
- High
- Medium
- Low
- Confirmed valid
- Confirmed invalid
- Validation error
- No validator
- Public
- Private
- Unknown
2. Under **Branches**, select a branch from the drop-down menu. | | View findings of a specific type of secret, such as **personal token** or **password**. | Under **Type**, select a type of secret. | | View findings of a specific severity | Under **Severity**, select a value. | You can triage findings in bulk by performing the following steps:
- **Undetermined**: no analysis available
- **Reachable**: detects vulnerable code patterns
- **Always reachable**: flags packages that are considered vulnerable regardless of usage
- **Conditionally reachable**: requires manual review and triage to determine impact
- **Malicious**: identifies dangerous packages or dangerous versions of a package.
- Always reachable
- Reachable
- Conditionally reachable
- Unreachable
- No reachability analysis
- Critical
- High
- Medium
- Low
Upgrade available - Upgrade unavailable
- Direct
- Transitive
- High
- Medium
- Low
- None
| Language | Supported package managers | Manifest file or lockfile |
|---|---|---|
| C# | NuGet | .csproj |
| Go | Go modules (go mod) |
go.mod |
| Java | Gradle | gradle.lockfile or build.gradle or
build.gradle.kts through Dynamic
Dependency Resolution. |
| Maven | Maven-generated dependency tree (see Setting up SSC scans for Apache Maven for instructions) or pom.xml through Dynamic
Dependency Resolution. |
|
| JavaScript or TypeScript | npm | package-lock.json |
| Yarn | yarn.lock |
|
| pnpm | pnpm-lock.yaml |
|
| Kotlin | Gradle | gradle.lockfile |
| Maven | Maven-generated dependency tree (See Setting up SSC scans for Apache Maven for instructions.) | |
| Python | pip |
|
| pip-tools | ||
| Pipenv | Pipfile.lock |
|
| Poetry | poetry.lock |
|
| uv | uv.lock |
|
| Ruby | RubyGems | Gemfile.lock |
| Scala | Maven | Maven-generated dependency tree (See Setting up SSC scans for Apache Maven for instructions.) |
| Swift | SwiftPM | Package.swift file and Swift-generated Package.resolved file. (See Swift documentation for instructions.) |
| Rust | Cargoβ‘ | cargo.lock |
| Dart | Pub | pubspec.lock |
| Elixir | Hex | mix.lock |
| PHP | Composer | composer.lock |
*β‘Supply Chain does not analyze the transitivity of packages for these language and manifest file or lockfile combinations. All dependencies are listed as **No
## Feature availability The following sections discuss the features supported by Supply Chain and available to you based on your project's language, package manager, and lockfile or manifest file availability. For projects with lockfiles, Semgrep parses lockfiles for dependencies, then scans your codebase for reachable findings based on the lockfiles. For a lockfile to be scanned by Semgrep Supply Chain, it must have one of the supported lockfile names. For some languages, a lockfile or manifest file is required to determine accurately
| Language | Reachability (see CVE coverage) |
Dynamic dependency resolution (beta) | License detection | Malicious dependency detection |
|---|---|---|---|---|
| C# | β | β
CI and CLI only |
β | β |
| Go | β | -- | β | β |
| Java | β | β | β | -- |
| JavaScript or TypeScript | β | -- | β | β |
| Kotlin | β | β | β | -- |
| Python | β | β
setup.py in CLI or CI |
β
For PyPi only |
β |
| Ruby | β | -- | β | β |
| Scala | β | β
SBT in CLI or CI |
β | -- |
| Swift | β | -- | β β | -- |
| PHP | β | -- | β | -- |
| Rust | No reachability analysis. However, Semgrep can compare a package's version against a list of versions with known vulnerabilities. | -- | β | β |
| Dart | -- | -- | -- | |
| Elixir | -- | -- | -- |
- Upgrade filter for Findings
- Upgrade guidance on the
Finding Details page - Coupled or blocked upgrade information shown on the
Finding Details page - Ability to open a PR to upgrade
- Upgrade filter for Findings
- Upgrade guidance on the
Finding Details page - Coupled or blocked upgrade information shown on the
Finding Details page - Ability to open a PR to upgrade
- Ability to open a PR to upgrade
- Upgrade filter for Findings
- Upgrade guidance on the
Finding Details page - Coupled or blocked upgrade information shown on the
Finding Details page
- Upgrade filter for Findings
- Upgrade guidance on the
Finding Details page - Coupled or blocked upgrade information shown on the
Finding Details page
Supports 35+ languages | **Semgrep Supply Chain**
Supports 14 languages | | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | [C#](/languages/csharp) | **Generally available**
β’ Cross-file dataflow analysis
β’ Supports up to C# 13
β’ 170+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses
β’ Can detect malicious dependencies | | [Go](/languages/go) | **Generally available**
β’ Cross-file dataflow analysis
β’ 80+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses
β’ Can detect malicious dependencies | | [Java](/languages/java) | **Generally available**
β’ Cross-file dataflow analysis
β’ Framework-specific control flow analysis
β’ 190+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses | | [JavaScript](/languages/javascript) | **Generally available**
β’ Cross-file dataflow analysis
β’ Framework-specific control flow analysis
β’ 250+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses
β’ Can detect malicious dependencies | | [Kotlin](/languages/kotlin) | **Generally available**
β’ Cross-file dataflow analysis
β’ 60+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses | | [Python](/languages/python) | **Generally available**
β’ Cross-file dataflow analysis
β’ Framework-specific control flow analysis
β’ 710+ Pro rules
β’ See [Python-specific support details](/languages/python) | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses
β’ Can detect malicious dependencies | | Typescript | **Generally available**
β’ Cross-file dataflow analysis
β’ Framework-specific control flow analysis
β’ 230+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect malicious dependencies
β’ Can detect open source licenses | | C / C++ | **Generally available**
β’ Cross-file dataflow analysis
β’ 150+ Pro rules | N/a | | JSX | **Generally available**
β’ Cross-function dataflow analysis
β’ 70+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses | | [Ruby](/languages/ruby) | **Generally available**
β’ Cross-function dataflow analysis
β’ 40+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses
β’ Can detect malicious dependencies | | [Scala](/languages/scala) | **Generally available**
β’ Cross-function dataflow analysis
β’ Community rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses | | [Swift](/languages/swift) | **Generally available**
β’ Cross-function dataflow analysis
β’ 60+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses | | Rust | **Generally available**
β’ Cross-function dataflow analysis
β’ 40+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses
β’ Can detect malicious dependencies | | PHP | **Generally available**
β’ Cross-function dataflow analysis
β’ 50+ Pro rules | **Generally available**
β’ Reachability analysis
β’ Can detect open source licenses | | Terraform | **Generally available**
β’ Cross-function dataflow analysis
β’ Community rules | N/a | | Generic | **Generally available** | N/a | | JSON | **Generally available** | N/a | | Elixir | **Generally available** | **Beta** | | APEX | **Beta** | -- | | Dart | **Experimental** | **Beta** |
When running `semgrep ci`, Semgrep fetches rules and any other configurations specific to your CI environment. Setting `SEMGREP_REPO_NAME` is optional, but ensures that: * Results are sent to the same project (either a repository or folder in a monorepo) in Semgrep AppSec Platform. * Any project-specific configurations, such as file ignores, are also respected. ## Troubleshooting GitHub The first piece of information that the team at Semgrep uses are the **GitHub Actions logs**. To retrieve a log, perform the following steps:
Extract the matched content as raw source code. This is the default value. `concat_json_string_array`
Extract the matched content as a JSON array. Each element of the array correspond to a line the resulting source code. This value is useful in extracting code from JSON formats such as Jupyter Notebooks. #### `reduce` The `reduce` key is optional in extract mode. The value of this key specifies a method to combine the ranges extracted by a single rule within a file. The value of `reduce` key must be one of the following: `separate`
Treat all matched ranges as separate units for subsequent queries. This is the **default** value. `concat`
Concatenate all matched ranges together and treat this result as a single unit for subsequent queries. ### Limitations of extract mode Although extract mode supports JSON array decoding with the `json` key, it does not support other additional processing for the extracted text, such as unescaping strings. While extract mode can help to enable rules which try and track taint across a language boundary within a file, taint rules cannot have a source and sink split across the original file and extracted text. ## Turbo Mode
`server = example.com`. In generic mode, an ellipsis extends until the end of the current block or up to 10 lines below, whichever comes first. To prevent this behavior, continue with the next step. 2. In the Semgrep rule, specify the following key: ```yaml theme={null} generic_ellipsis_max_span: 0 ``` This option forces the ellipsis operator to match patterns within a single line. Example of the [resulting rule](https://semgrep.dev/playground/s/KPzn): ```yaml theme={null} id: password-in-config-file pattern: | password = $...PASSWORD options: # prevent ellipses from matching multiple lines generic_ellipsis_max_span: 0 message: | password found in config file: $...PASSWORD languages: - generic severity: WARNING ``` ### Ignoring comments By default, the generic mode does **not** know about comments or code that can be ignored. The following example is scanning for CSS code that sets the text color to blue. The target code is the following: ``` color: /* my fave color */ blue; ``` Use the [`options.generic_comment_style`](/writing-rules/rule-syntax/#options) to ignore C-style comments, as is the case in the example. The Semgrep rule is: ```yaml theme={null} id: css-blue-is-ugly pattern: | color: blue options: # ignore comments of the form /* ... */ generic_comment_style: c message: | Blue is ugly. languages: - generic severity: WARNING ``` ## Command line example Sample pattern: `exec(...)` Sample target file `exec.txt` contains: ```bash theme={null} import exec as safe_function safe_function(user_input) exec("ls") exec(some_var) some_exec(foo) exec (foo) exec ( bar ) # exec(foo) print("exec(bar)") ``` Output: ```bash theme={null} $ semgrep -l generic -e 'exec(...)` exec.text 7:exec("ls") -------------------------------------------------------------------------------- 11:exec(some_var) -------------------------------------------------------------------------------- 19:exec (foo) -------------------------------------------------------------------------------- 23:exec ( 24:128 25: Β Β bar 26:129 27:) -------------------------------------------------------------------------------- 31:# exec(foo) -------------------------------------------------------------------------------- 35:print("exec(bar)") ran 1 rules on 1 files: 6 findings ``` ## Semgrep Registry rules for generic pattern matching You can peruse [existing generic rules](https://semgrep.dev/r?lang=generic\&sev=ERROR,WARNING,INFO\&tag=dgryski.semgrep-go,hazanasec.semgrep-rules,ajinabraham.njsscan,best-practice,security,java-spring,go-stdlib,ruby-stdlib,java-stdlib,js-node,nodejsscan,owasp,dlint,react,performance,compatibility,portability,correctness,maintainability,security,mongodb,experimental,caching,robots-denied,missing-noreferrer,missing-noopener) in the Semgrep registry. In general, short patterns on structured data performs the best. ## Cheat sheet Some examples of what matches and what doesn't match on the `generic` tab of the Semgrep cheat sheet below: ## Hidden bonus In the Semgrep code, the generic pattern matching implementation is called **spacegrep** because it tokenizes based on whitespace (and because it sounds cool π). # Static analysis and rule-writing glossary Source: https://docs.semgrep.dev/writing-rules/glossary The definitions provided here are specific to Semgrep. ## Constant propagation Constant propagation is a type of analysis where values known to be constant are substituted in later uses, allowing the value to be used to detect matches. Semgrep can perform constant propagation across files, unless you are running Semgrep Community Edition (CE), which can only propagate within a file. Constant propagation is applied to all rules unless [it is disabled](/writing-rules/data-flow/constant-propagation#disable-constant-propagation). For example, given the following pattern: ```yaml theme={null} ... patterns: - pattern: console.log(2) ``` And the following code snippet: ```javascript highlight={2} theme={null} const x = 2; console.log(x); ``` The pattern operator `pattern: print(2)` tells Semgrep to match line 2 because it propagates the value `2` from the assignment in line 1 to the `console.log()` function in line. Constant propagation is one of the many analyses that differentiate Semgrep from grep. ## Cross-file analysis Cross-file analysis (also known as **interfile analysis**) takes into account how information flows between files. In particular, cross-file analysis includes **cross-file taint analysis**, which tracks unsanitized variables flowing from a source to a sink through arbitrarily many files. Other analyses performed across files include constant propagation and type inference. Cross-file analysis is usually used in contrast to intrafile, or per-file analysis, where each file is analyzed as a standalone block of code. Within Semgrep, cross-file **and** cross-function analysis is simply referred to as cross-file analysis. Semgrep CE is limited to per-file analysis. ## Cross-function analysis Cross-function analysis means that interactions between functions are taken into account. This improves taint analysis, which tracks unsanitized variables flowing from a source to a sink through arbitrarily many functions. Within Semgrep documentation, cross-function analysis implies intrafile or per-file analysis. Each file is still analyzed as a standalone block, but within the file it takes into account how information flows between functions. Also known as **interprocedural** analysis. ## Error matrix An error matrix is a 2x2 table that visualizes the findings of a Semgrep rule in relation to the vulnerable lines of code it does or doesn't detect. It has two axes: * Positive and negative * True or false These yield the following combinations: **True positive**
Β Β Β Β The rule detected a piece of code it was intended to find. **False positive**
Β Β Β Β The rule detected a piece of code it was not intended to find. **True negative**
Β Β Β Β The rule correctly skipped over a piece of code it wasn't meant to find. **False negative**
Β Β Β Β The rule failed to detect a piece of code it should have found. Not to be confused with **risk matrices**. ## Finding A finding is the core result of Semgrep's analysis. Findings are generated when a Semgrep rule matches a piece of code. Findings can be security issues, bugs, or code that doesn't follow coding conventions. ## Fully qualified name A **fully qualified name** refers to a name which uniquely identifies a class, method, type, or module. Languages such as C# and Ruby use `::` to distinguish between fully qualified names and regular names. Not to be confused with **tokens**. ## l-value (left-, or location-value) An expression that denotes an object in memory; a memory location, something that you can use in the left-hand side (LHS) of an assignment. For example, `x` and `array[2]` are l-values, but `2+2` is not. ## Metavariable A metavariable is an abstraction that lets you match something even when you don't know exactly what it is you want to match. It is similar to capture groups in regular expressions. All metavariables begin with a `$` and can only contain uppercase characters, digits, and underscores. ## Propagator A propagator is any code that alters a piece of data as the data moves across the program. This includes functions, reassignments, and so on. When you write rules that perform taint analysis, propagators are pieces of code that you specify through the `pattern-propagator` key as code that always passes tainted data. This is especially relevant when Semgrep performs intraprocedural taint analysis, as there is no way for Semgrep to infer which function calls propagate taint. Thus, explicitly listing propagators is the only way for Semgrep to know if tainted data could be passed within your function. ## Rule (Semgrep rule) A rule is a specification of the patterns that Semgrep must match to the code to generate a finding. Rules are written in YAML. Without a rule, the engine has no instructions on how to match code. Rules can be run on either Semgrep or its OSS Engine. Only proprietary Semgrep can perform [interfile analysis](#cross-file-analysis). There are two types of rules: **search** and **taint**. **Search rules**
Rules default to this type. Search rules detect matches based on the patterns described by a rule. There are several semantic analyses that search rules perform, such as: * Interpreting syntactically different code as semantically equivalent * Constant propagation * Matching a fully qualified name to its reference in the code, even when not fully qualified * Type inference, particularly when using typed metavariables **Taint rules**
Taint rules make use of Semgrep's taint analysis in addition to default search functionalities. Taint rules are able to specify sources, sinks, and propagators of data as well as sanitizers of that data. For more information, see [Taint analysis documentation](/writing-rules/data-flow/taint-mode/overview). ## Sanitizers A sanitizer is any piece of code, such as a function or [a cast](https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/types/casting-and-type-conversions#explicit-conversions), that can clean untrusted or tainted data. Data from untrusted sources, such as user inputs, may be tainted with unsafe characters. Sanitizers ensure that unsafe characters are removed or stripped from the input. An example of a sanitizer is the [
- `c` for traditional C-style comments (`/* ... */`).
- `cpp` for modern C or C++ comments (`// ...` or `/* ... */`).
- `shell` for shell-style comments (`# ...`).
`. |
The complete list of available options can be consulted in the [Semgrep matching engine configuration](https://github.com/semgrep/semgrep/blob/develop/interfaces/Rule_options.atd) module. Please note that options not included in the table above are considered experimental and may change or be removed without notice.
## `fix`
The `fix` top-level key allows simple pattern fixes by suggesting an alternative for each match. Run `semgrep` with `--autofix` to apply the changes to the files.
Example:
```yaml theme={null}
rules:
- id: use-dict-get
patterns:
- pattern: $DICT[$KEY]
fix: $DICT.get($KEY)
message: "Use `.get()` method to avoid a KeyNotFound error"
languages: [python]
severity: HIGH
```
For more information about `fix` and `--autofix` see [Rule-defined fix](/writing-rules/rule-defined-fix) documentation.
## `metadata`
Provide additional information for a rule with the `metadata:` key, such as a related CWE, likelihood, or OWASP.
Example:
```yaml theme={null}
rules:
- id: eqeq-is-bad
patterns:
- [...]
message: "useless comparison operation `$X == $X` or `$X != $X`"
metadata:
cve: CVE-2077-1234
discovered-by: Ikwa L'equale
languages:
- javascript
- python
- go
severity: MEDIUM
```
The metadata are also displayed in the output of Semgrep if youβre running it with `--json`.
Rules with `category: security` have additional metadata requirements. See [Including fields required by security category](/contributing/contributing-to-semgrep-rules-repository/#fields-required-by-the-security-category) for more information.
## `min-version` and `max-version`
Each rule supports optional fields `min-version` and `max-version` specifying
minimum and maximum Semgrep versions. If the Semgrep
version being used doesn't satisfy these constraints,
the rule is skipped without causing a fatal error.
Example rule:
```yaml theme={null}
rules:
- id: bad-goflags
# earlier semgrep versions can't parse the pattern
min-version: 1.31.0
pattern: |
ENV ... GOFLAGS='-tags=dynamic -buildvcs=false' ...
languages: [dockerfile]
message: "We should not use these flags"
severity: MEDIUM
```
Another use case is when a newer version of a rule works better than
before but relies on a new feature. In this case, you can use
`min-version` and `max-version` to ensure that either the older or the
newer rule is used, but not both. The rules would look like this:
```yaml theme={null}
rules:
- id: something-wrong-v1
max-version: 1.72.999
...
- id: something-wrong-v2
min-version: 1.73.0
# 10x faster than v1!
...
```
The `min-version`/`max-version` feature has been available since Semgrep 1.38.0. It is intended primarily for publishing rules that rely on
newly released features without causing errors in older Semgrep
installations.
## `category`
Provide a category for users of the rule. For example: `best-practice`, `correctness`, `maintainability`. For more information, see [Semgrep Registry rule requirements](/contributing/contributing-to-semgrep-rules-repository/#semgrep-registry-rule-requirements).
## `paths`
### Exclude a rule in paths
To ignore a specific rule on specific files, set the `paths:` key with
one or more filters. The patterns apply to the full file paths
relative to the project root.
Example:
```yaml theme={null}
rules:
- id: eqeq-is-bad
languages:
- python
- javascript
severity: MEDIUM
pattern: $X == $X
paths:
exclude:
- "src/**/*.jinja2"
- "*_test.go"
- "project/tests"
- "project/static/*.js"
```
When invoked with `semgrep -f rule.yaml project/`, the preceding rule runs on files inside `project/`, but no results are returned for:
* any file with a `.jinja2` file extension
* any file whose name ends in `_test.go`, such as `project/backend/server_test.go`
* any file inside `project/tests` or its subdirectories
* any file matching the `project/static/*.js` glob pattern
**NOTE**
The glob syntax is from [Python's `wcmatch`](https://pypi.org/project/wcmatch/) and is used to match against the given file and all its parent directories.
### Limit a rule to paths
Conversely, to run a rule *only* on specific files, set a `paths:` key with one or more of these filters:
```yaml theme={null}
rules:
- id: eqeq-is-bad
pattern: $X == $X
languages:
- python
- javascript
severity: MEDIUM
paths:
include:
- "*_test.go"
- "project/server"
- "project/schemata"
- "project/static/*.js"
- "tests/**/*.js"
```
When invoked with `semgrep -f rule.yaml project/`, this rule runs on files inside `project/`, but results are returned only for:
* files whose name ends in `_test.go`, such as `project/backend/server_test.go`
* files inside `project/server`, `project/schemata`, or their subdirectories
* files matching the `project/static/*.js` glob pattern
* all files with the `.js` extension, arbitrary depth inside the tests folder
If you are writing tests for your rules, add any test file or directory to the included paths as well.
**NOTE**
When mixing inclusion and exclusion filters, the exclusion ones take precedence.
Example:
```yaml theme={null}
paths:
include: "project/schemata"
exclude: "*_internal.py"
```
The preceding rule returns results from `project/schemata/scan.py` but not from `project/schemata/scan_internal.py`.
## Additional examples
This section contains more complex rules that perform advanced code searching.
### Complete useless comparison
```yaml theme={null}
rules:
- id: eqeq-is-bad
languages: [python]
severity: MEDIUM
patterns:
- pattern-not-inside: |
def __eq__(...):
...
- pattern-not-inside: assert(...)
- pattern-not-inside: assertTrue(...)
- pattern-not-inside: assertFalse(...)
- pattern-either:
- pattern: $X == $X
- pattern: $X != $X
- patterns:
- pattern-inside: |
def __init__(...):
...
- pattern: self.$X == self.$X
- pattern-not: 1 == 1
message: "useless comparison operation `$X == $X` or `$X != $X`"
```
The preceding rule makes use of many operators. It utilizes `pattern-either`, `patterns`, `pattern`, and `pattern-inside` to carefully consider different cases, and employs `pattern-not-inside` and `pattern-not` to exclude specific unnecessary comparisons.
## Full specification
The [full configuration-file format](https://github.com/semgrep/semgrep-interfaces/blob/main/rule_schema_v1.yaml) is defined as a [jsonschema](http://json-schema.org/specification.html) object.
# Test rules
Source: https://docs.semgrep.dev/writing-rules/testing-rules
Semgrep provides a testing mechanism for your rules. You can write code and provide annotations to let Semgrep know where you are or aren't expecting findings. Semgrep provides the following annotations:
* `ruleid: ` for protecting against false negatives
* `ok: ` for protecting against false positives
* `todoruleid: ` for future "positive" rule improvements
* `todook: ` for future "negative" rule improvements
When writing tests, remember that:
1. The `--test` flag tells Semgrep to run tests in the specified directory.
2. Annotations are specified as a comment immediately preceding the offending line.
3. Semgrep looks for tests based on the rule filename and the languages
specified in the rule. In other words, `path/to/rule.yaml` searches for
`path/to/rule.py`, `path/to/rule.js`, and similar, based on the languages specified in the rule.
**INFO**
The `.test.yaml` file extension can also be used for test files. This is necessary when testing YAML language rules.
## Test rules with Rule-defined fix
Semgrep's testing mechanism also provides a way to test the behavior of any `fix` values defined in the rules.
To define a test for Rule-defined fix behavior:
Create a new **Rule-defined fix test file** with the `.fixed` suffix before the file type extension. For example, name the Rule-defined fix test file of a rule with test code in `path/to/rule.py` as `path/to/rule.fixed.py`.
Within the Rule-defined fix test file, enter the expected result of applied Rule-defined fix rule to the test code.
Run `semgrep --test` to verify that your Rule-defined fix test file is correctly detected.
When you use `semgrep --test`, Semgrep applies the Rule-defined fix rule to the original test code (`path/to/rule.py`), then verifies whether this matches the expected outcome defined in the Rule-defined fix test file (`path/to/rule.fixed.py`). If there is a mismatch, the line diffs are printed.
**INFO**
**Hint**: Creating a Rule-defined fix test for a rule with Rule-defined fix can take less than a minute with the following flow of commands:
```sh theme={null}
cp rule.py rule.fixed.py
semgrep --config rule.yaml rule.fixed.py --autofix
```
These commands apply the Rule-defined fix to the test code. After Semgrep delivers a fix, inspect whether the outcome of this fix is as expected (for example, using `vimdiff rule.py rule.fixed.py`).
## Example
Consider the following rule:
```yaml theme={null}
rules:
- id: insecure-eval-use
patterns:
- pattern: eval($VAR)
- pattern-not: eval("...")
fix: secure_eval($VAR)
message: Calling 'eval' with user input
languages: [python]
severity: MEDIUM
```
If the filename with the preceding rule is `rules/detect-eval.yaml`, you can create `rules/detect-eval.py`:
```python theme={null}
from lib import get_user_input, safe_get_user_input, secure_eval
user_input = get_user_input()
# ruleid: insecure-eval-use
eval(user_input)
# ok: insecure-eval-use
eval('print("Hardcoded eval")')
totally_safe_eval = eval
# todoruleid: insecure-eval-use
totally_safe_eval(user_input)
# todook: insecure-eval-use
eval(safe_get_user_input())
```
Run the tests with the following:
```sh theme={null}
semgrep --test rules/
```
This produces the following output:
```sh theme={null}
1/1: β All tests passed
No tests for fixes found.
```
Semgrep tests automatically avoid failing on lines marked with `# todoruleid` or `# todook`.
## Store rules and test targets in different directories
Creating different directories for rules and tests helps you manage a growing library of custom rules. To store rules and test targets in different directories, use the `--config` option.
For example, in the directory with the following structure:
```sh theme={null}
$ tree tests
tests
βββ rules
βΒ Β βββ python
βΒ Β βββ insecure-eval-use.yaml
βββ targets
βββ python
βββ insecure-eval-use.py
4 directories, 2 files
```
Use of the following command
```sh theme={null}
semgrep --test --config tests/rules/ tests/targets/
```
Produces the same output as in the previous example.
The subdirectory structure of these two directories must be the same for Semgrep to correctly find the associated files.
To test the Rule-defined fix behavior, add the Rule-defined fix test file `rules/detect-eval.fixed.py` to represent the expected outcome of applying the fix to the test code:
```python theme={null}
from lib import get_user_input, safe_get_user_input, secure_eval
user_input = get_user_input()
# ruleid: insecure-eval-use
secure_eval(user_input)
# ok: insecure-eval-use
eval('print("Hardcoded eval")')
totally_safe_eval = eval
# todoruleid: insecure-eval-use
totally_safe_eval(user_input)
# todook: insecure-eval-use
secure_eval(safe_get_user_input())
```
So that the directory structure is printed as the following:
```sh theme={null}
$ tree tests
tests
βββ rules
βΒ Β βββ python
βΒ Β βββ insecure-eval-use.yaml
βββ targets
βββ python
βββ insecure-eval-use.py
βββ insecure-eval-use.fixed.py
4 directories, 2 files
```
Use of the following command:
```sh theme={null}
semgrep --test --config tests/rules/ tests/targets/
```
Results in the following outcome:
```sh theme={null}
1/1: β All tests passed
1/1: β All fix tests passed
```
If the fix does not behave as expected, the output prints a line diff.
For example, if you replace `secure_eval` with `safe_eval`, you can see that lines 5 and 15 do not render as expected.
```sh theme={null}
1/1: β All tests passed
0/1: 1 fix tests did not pass:
--------------------------------------------------------------------------------
β targets/python/detect-eval.fixed.py <> autofix applied to targets/python/detect-eval.py
---
+++
@@ -5 +5 @@
-safe_eval(user_input)
+secure_eval(user_input)
@@ -15 +15 @@
-safe_eval(safe_get_user_input())
+secure_eval(safe_get_user_input())
```
## Validating rules
You can run `semgrep --validate --config [filename]` to verify the rule's configuration. This command runs a combination of Semgrep rules and OCaml checks against your rules to search for issues such as duplicate patterns and missing fields. All rules submitted to the Semgrep Registry are validated.
The semgrep rules are pulled from `p/semgrep-rule-lints`.
This feature is still experimental and under active development. Your feedback is welcomed!
## Enable Rule-defined fix in Semgrep Code
To enable Rule-defined fix for all projects in your Semgrep organization:
In Semgrep AppSec Platform, go to [**Settings > General > Code**](https://semgrep.dev/orgs/-/settings/general/code).
Click the **Rule-defined fix** toggle to enable this feature.