Documentation Index

Fetch the complete documentation index at: https://docs.semgrep.dev/llms.txt

Use this file to discover all available pages before exploring further.

• the principles that guide our data-collection decisions
• how to change when Semgrep sends metrics
• what data is not collected
• what data is collected

​

Semgrep metrics

• the principles that guide our data-collection decisions
• how to change when Semgrep sends metrics
• what data is not collected
• what data is collected

​

Principles

1. Transparency: Collect and use data in a way that is clearly explained to the user and benefits them
2. User control: Put users in control of their data at all times
3. Limited data: Collect what is needed, pseudoanonymize where possible, and delete when no longer necessary

​

Automatic collection, opt-in, and opt-out

$ semgrep --config=myrule.yaml  # → no metrics (loading rules from local file)
$ semgrep --config=p/python     # → metrics enabled (fetching Registry)
$ semgrep login && semgrep ci   # → metrics enabled (logged in to semgrep.dev)

• --metrics auto: (default) metrics are sent whenever rules are pulled from the Semgrep Registry or the user is logged in.
• --metrics on: metrics are sent on every Semgrep run
• --metrics off: metrics are never sent

​

Data NOT collected

​

Data NOT collected ever

• Source code
• Private rules

​

Data NOT collected unless explicitly requested

• Filenames
• Git commit hashes, timestamps, messages, authors
• User-identifiable data about Semgrep’s findings in your code, including finding messages

​

Data collected as metrics

​

Environmental

• How long the command took to run
• The version of Semgrep CLI
• An anonymous user ID that identifies the machine
• IP address that triggers a run
• Value of the CI environment variable, if set
• Pseudoanonymized hash of the scanned project’s name
• Pseudoanonymized hash of the rule definitions run
• Pseduoanonymized hash of the config option. (Note that when a config option downloads a ruleset from the https://semgrep.dev registry, feature usage metrics still include the ruleset name in plain text.)

​

Performance

• Runtime duration
• Duration of individual phases (e.g. parsing)
• Total number of rules
• Total number of files
• Project size in bytes

​

Parse Rates

• Number of targeted files
• Number of files without any parse-related error
• Number of bytes across targeted files
• Number of bytes without any parse-related error

​

Errors

• Semgrep’s return code
• The number of errors
• Compile-time error names, e.g., MaxFileSizeExceeded, SystemOutOfMemory, UnknownFileEncoding

​

Value

• Number of raised findings
• Number of ignored findings
• Pseudoanonymized hashes of the rule definitions that yield findings
• The features used during the scan
• The engine type requested for the scan

​

Extension

• IDE being used
• Version of IDE integration
• Number of fixes triggered through the integration
• Number of findings ignored through the integration

​

Pseudoanonymization

• Track performance and value improvements over successive runs on projects
• Remove test data from our metrics

• An entity that independently knows the value of an input datum AND who has access to Semgrep, Inc’s metrics data could access metrics for that known datum

• Treat collected metrics data as secret, using application-security best practices, including (but not limited to)
  • Encryption during transit and rest
  • Strict access control to data-storage systems
  • Application-security-policy requirements for third parties (e.g. cloud-service providers; see “data sharing” below)
• Only correlate hashed data to input data when these inputs are already known to Semgrep, Inc (e.g. publicly available project URLs for open-source projects, or projects that log in to the Semgrep Registry)

​

Description of metrics fields

​

Anonymous user ID

anonymous_user_id: "5f52484c-3f82-4779-9353-b29bbd3193b6"

• How many people use a given rule/ruleset/snippet?

  This enables the Semgrep team to assess their performance, and we’re planning to make these numbers public for all rule authors in the community.

• What percentage of users log in?

  We use this to evaluate our success as we build new authenticated features for the Semgrep Cloud Platform.

• How often are individual subcommands and CLI features used?

  This helps our product and developer experience teams measure feature adoption rate, analyze anonymized usage, and compare cohort behavior to improve our product offerings.

​

Feature usage

"features": ["language/python", "option/deep", "option/no-git-ignore", "key/metavariable-comparison"]

• language: What languages were scanned
• cli-flag/cli-envvar: What options were configured (does NOT include their value)
• config: What method was used to retrieve rules (does NOT include any of the rule)
• registry-query: The value of a --config r/foo.bar.baz setting, limited to the first word (e.g. r/foo.. in this example)
• ruleset: The value of a --config p/foobar setting
• semgrepignore: Whether an :include statement was used in a .semgrepignore file
• subcommand: What subcommand was used (e.g. scan or ci)

• How many people use a given feature?

  This guides our development, and lets us decide when and how to deprecate features.

• How does feature usage affect finding counts, error counts, and performance?

  We use this to evaluate experimental features and understand their production-readiness.

Engine requested (OSS, Pro, Interfile)

​

Sample metrics

{
    "started_at": "2021-05-10T21:05:06+00:00",
    "sent_at": "2021-05-10T21:05:09+00:00",
    "event_id": "222bcccd-9dc2-4d10-ac3a-5692460e77ee",
    "anonymous_user_id": "5f52484c-3f82-4779-9353-b29bbd3193b6",
    "environment": {
        "version": "0.51.0",
        "ci": "true",
        "configNamesHash": "ede96c41b57de3e857090fb3c486e69ad8efae3267bac4ac5fbc19dde7161094",
        "projectHash": "c65437265631ab2566802d4d90797b27fbe0f608dceeb9451b979d1671c4bc1a",
        "rulesHash": "b03e452f389e5a86e56426c735afef13686b3e396499fc3c42561f36f6281c43",
        "isAuthenticated": false
    },
    "performance": {
        "runTime": 37.1234233823,
        "numRules": 2,
        "numTargets": 573,
        "totalBytesScanned": 33938923,
        "ruleStats": [{
          "ruleHash": "7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166",
          "parseTime": 0,
          "matchTime": 0.05480456352233887,
          "runTime": 0.20836973190307617,
          "bytesScanned": 0
        }],
        "fileStats": [{
          "size": 6725,
          "numTimesScanned": 147,
          "parseTime": 0.013289928436279297,
          "matchTime": 0.05480456352233887,
          "runTime": 0.20836973190307617
        }]
    },
    "parse_rate": {
      "python": {
        "num_targets": 102,
        "targets_parsed": 101,
        "num_bytes": 985123,
        "bytes_parsed": 993419
      },
      "ruby": {
        "num_targets": 12,
        "targets_parsed": 12,
        "num_bytes": 341027,
        "bytes_parsed": 341027
      }
    },
    "errors": {
        "returnCode": 1,
        "errors": ["UnknownLanguage"],
        "warnings": ["MaxFileSizeExceeded", "TimeoutExceeded"]
    },
    "value": {
        "ruleHashesWithFindings": {"7c43c962dfdbc52882f80021e4d0ef2396e6a950867e81e5f61e68390ee9e166": 4},
        "numFindings": 7,
        "numIgnored": 3,
        "features": ["language/python", "option/deep", "option/no-git-ignore", "key/metavariable-comparison"],
        "engineRequested": "OSS",
        "engineConfig": { analysis_type: "Intraprocedural", pro_langs: false }
    }
}

​

Data collected when explicitly requested

• Project identity (e.g., name, URL)
• Scan environment (e.g., CI provider, OS)
• Author identity (e.g., committer email)
• Commit metadata (e.g., commit hash and timestamp)
• Review and review-requester identifying data (e.g., pull-request ID, branch, merge base, request author)
• Scan metadata, including type of scan and scan parameters (e.g., paths scanned and extensions of ignored files)
• Timing metrics (e.g., time taken to scan per-rule and per-path)
• Parse metrics (e.g., number of files targeted and parsed per-language)
• Semgrep CLI environment (e.g., version, interpreter, timestamp)

• Check ID and metadata (as defined in the rule definition; e.g., OWASP category, message, severity)
• Code location, including file path, that triggered findings
• A one-way hash of a unique code identifier that includes the triggering code content
• Dependency name and version (only sent when using Semgrep Supply Chain)
• Source code is NOT collected

• Package name (e.g., lodash)
• Package version (e.g., 1.2.3)
• File path for lockfile (e.g., frontend/yarn.lock)
• Analysis of external dependency calls. (e.g., from flask import Response, Response(status=204))

​

Debugging data collected when traces are requested

1. Info (--trace): basic tracing. Sends basic info about settings passed via environment variables or flags, and timings around scan phases.
2. Debug (--trace with SEMGREP_TRACE_LEVEL=debug): debug tracing. Sends additional timings, particularly around functions run during taint analysis, timings about each file as it undergoes pre-processing and then matching. Includes the file path and sometimes rule names.
3. Trace (--trace with SEMGREP_TRACE_LEVEL=trace): even more detailed debug tracing.

• Semgrep function currently running (e.g. Match_tainting_mode.check_rules)
• Start time (e.g. 1718775054055113)
• Duration (e.g. 934956)
• Settings passed (e.g. --jobs, --timeout)

• Repo name (e.g. semgrep-app)
• Number of matches (e.g. 2)
• Number of errors (e.g. 1)
• Number of rules (e.g. 12)
• Number of targets (e.g. 128)
• Is a diff scan (e.g. false)
• Is an interfile scan (e.g. true)
• Cryptographic hash of rules (e.g. One-way hash of the rule definitions)
• Other scan settings of a similar nature. Summary data will only include information that semgrep ci has access to.

• Stacktraces from when Semgrep crashes
• Warnings about high memory usage
• Informational logs about which stage of scanning Semgrep is performing

• file paths (e.g. src/my_file.py)
• Hashed function names (e.g. d40fdc8ef9bf7b7dd1b014533a58a05e9b98d7dd856784352201388fe5e22673)
• Hashed variable names (e.g. 0268934f5c43d1b5fc7d52d9efe17c69f1144b108c384c3513cbe493043712b3)

​

Registry fetches

​

Data sharing