Skip to content

PR Scanning Strategy

PR scanning gives developers security feedback before code reaches the main branch. This guide covers how to design a PR scanning strategy that balances security with developer velocity.

Goals

A good PR scanning strategy:

  1. Catches real issues before they reach production
  2. Doesn't block developers on false positives or low-severity noise
  3. Provides fast feedback — ideally under 2 minutes
  4. Gives actionable guidance — not just "you have a vulnerability"

Scanner selection for PRs

PR scans should be fast. Choose scanners accordingly:

Scanner PR scan time Recommended for PRs?
Gitleaks 5-15 seconds Yes — always include
Grype 10-30 seconds Yes — catches new vulnerable deps
Semgrep 30s - 3 minutes Yes, if speed is acceptable
Trivy (container) 30s - 2 minutes Only if Dockerfile is changed

Start with secrets + SCA

Gitleaks + Grype gives you fast, high-signal feedback. Add Semgrep once your team is comfortable with the workflow.

Gate strategy

Level 1 — Inform only (no blocking)

Post a comment with findings but don't block the PR:

package mayo.pr_scan

import rego.v1

default result := "pass"
default block := false

message := sprintf("Found %d new finding(s)", [
    input.scan_results.total_new
]) if {
    input.scan_results.total_new > 0
}

Best for: Teams new to PR scanning. Builds awareness without friction.

Level 2 — Block on critical only

Block PRs that introduce critical vulnerabilities:

package mayo.pr_scan

import rego.v1

result := "fail" if {
    input.scan_results.by_severity.critical > 0
}

default result := "pass"

Best for: Most teams. Catches the worst issues without excessive blocking.

Level 3 — Block on critical + high

package mayo.pr_scan

import rego.v1

result := "fail" if {
    input.scan_results.by_severity.critical + input.scan_results.by_severity.high > 0
}

default result := "pass"

Best for: Teams with mature triage policies and low false-positive rates.

Level 4 — Block on any new finding

package mayo.pr_scan

import rego.v1

result := "fail" if {
    input.scan_results.total_new > 0
}

default result := "pass"

Warning

This is very strict. Only use this for critical repositories with well-tuned scanners. High false-positive rates will frustrate developers.

Graduated rollout

Roll out PR scanning in phases:

Phase Duration Gate level Repositories
1 Week 1-2 Inform only 3-5 pilot repos
2 Week 3-4 Block on critical Pilot repos
3 Month 2 Block on critical All active repos
4 Month 3+ Block on critical+high High-risk repos

Comment styles

Summary comment

A single comment at the top of the PR:

## Mayo ASPM Scan Results

| Severity | New | Fixed |
|----------|-----|-------|
| Critical | 0   | 0     |
| High     | 1   | 0     |
| Medium   | 3   | 1     |

**1 blocking finding** — [View details](https://mayoaspm.com/...)

Inline annotations

Comments on specific lines in the PR diff:

⚠️ SQL Injection (high)
semgrep: java.lang.security.audit.sqli.tainted-sql-string
This query uses unsanitized user input. Use parameterized queries instead.

Both

Use summary + inline for the best developer experience.

Handling edge cases

Draft PRs

Skip scanning on draft PRs to save resources:

result := "pass" if {
    input.pull_request.draft == true
}

Dependabot / Renovate PRs

Auto-generated dependency update PRs should still be scanned but may warrant different rules:

result := "fail" if {
    input.pull_request.author in ["dependabot[bot]", "renovate[bot]"]
    input.scan_results.by_severity.critical > 0
}

Branch-specific rules

Stricter rules for PRs targeting main:

result := "fail" if {
    input.pull_request.base_branch == "main"
    input.scan_results.by_severity.critical + input.scan_results.by_severity.high > 0
}

result := "fail" if {
    input.pull_request.base_branch != "main"
    input.scan_results.by_severity.critical > 0
}

default result := "pass"

Performance tips

  1. Use differential scanning — Mayo ASPM only scans the PR diff, not the entire repo.
  2. Limit scanner count — 2-3 scanners for PRs is usually enough.
  3. Set scan timeouts — if a scan takes > 5 minutes, it may indicate an issue.
  4. Cache results — re-pushes to the same PR only scan changed files.

Metrics

Track PR scanning effectiveness:

Metric Target Description
PR scan time < 2 minutes Time from PR event to check run result
Block rate < 5% of PRs PRs blocked by scan failures
False block rate < 1% of PRs PRs blocked incorrectly
Developer override rate < 2% PRs merged despite scan failure

Next steps