Skip to main content
Grammar pins, Semgrep wrappers, and generated parsers live in the Semgrep repository. Run all commands from its root, which contains libs/, languages/, and src/.

​
Summary

1
Update the grammar’s commit and, if needed, Tree-sitter version in the registry.
2
Adjust the Semgrep wrapper and add regression tests.
3
Test the grammar with make test-grammar-<LANG>.
4
Regenerate each parser in the entry’s regen list with, for example, make regen-grammar-<LANG>.
5
Update the CST-to-AST conversion if needed, then run make core and make test.
6
Open a pull request against develop.

​
Before upgrading

Grammar development requires Node.js, Cargo, clang/clang++, and Git LFS on PATH. The examples below update Python. For wrapper-only changes, skip the first step.

​
1. Update the registry

Edit libs/ocaml-tree-sitter-semgrep/lang/upstream-grammars.json. Replace commit with the upstream commit SHA. Change tree_sitter if needed. The following example shows Python’s entry:
tree_sitter selects the grammar’s CLI version independently of the shared runtime version in libs/ocaml-tree-sitter-semgrep/core/tree-sitter-version.

​
2. Update the wrapper and regression tests

Adjust the Semgrep syntax extensions if needed. For Python, edit libs/ocaml-tree-sitter-semgrep/lang/semgrep-grammars/src/semgrep-python/grammar.js and add regression cases to libs/ocaml-tree-sitter-semgrep/lang/semgrep-grammars/src/semgrep-python/test/corpus/semgrep.txt. Include the grammar sources in libs/ocaml-tree-sitter-semgrep/lang/python/fyi.list. Paths in that file are relative to libs/ocaml-tree-sitter-semgrep/lang/:

​
3. Test the grammar

Test the wrapper’s Tree-sitter corpus and generated OCaml parser:
make test-grammar-python fetches the pinned upstream sources and rebuilds the wrapper.

​
4. Regenerate the parser

Run make regen-grammar-<dest> for every destination in the entry’s regen list:
make regen-grammar-python updates languages/python/tree-sitter/semgrep-python/, including generated lib/, bin/, config/, and fyi/ files. Change the wrapper and regenerate instead of editing generated files by hand. A registry key can produce multiple destinations: typescript produces both typescript and tsx. Test the key, then regenerate each destination:
If another entry lists the changed grammar in depends_on, also test and regenerate that consumer (for example, cpp after changing c). Entries with regen: [] do not produce a Semgrep parser.
Check generated CST code for Blank nodes, which lack token locations:
If any appear, give the corresponding grammar node a named rule and regenerate.

​
5. Build and test Semgrep

If the CST changed, update languages/python/tree-sitter/Parse_python_tree_sitter.ml, using languages/python/tree-sitter/semgrep-python/lib/Boilerplate.ml as a reference. Add regression tests under tests/parsing/python/ and tests/patterns/python/.

​
See also

How to add support for a new language