Skip to main content
SAICA-KG

Tools

pytest

12,000

Python test framework — discipline gate for "done means tests pass."

DetectionPost-generationFully autonomousStableTier 2AEnvironment
Maturity tier details

Computed tier: 2A(raw score 5.13). Effective tier: 2A.

Facets:documented_in=0evaluated_on=0integration_surfaces=1.5paper_citations=0recent_commit=2stars=1.63

Source of truth:pipeline/maturity/score.py

Description

pytest is the de facto Python test framework: it discovers and runs unit, integration, and functional tests, reporting failures in human-readable form. It is the canonical discipline gate for the "done means tests pass" rule — surfacing incomplete execution (skipped/never-written tests), test_manipulation (xfail/skip abuse), and logic errors that only show up under exercise. Configuration lives in `pyproject.toml [tool.pytest.ini_options]`, which the audit detector now recognises.

Why it's included

Occupies detection × post_generation × fully_autonomous for the incomplete_execution and test_manipulation failure modes in the Python lane. Mature workhorse; missing it leaves the audit detector unable to recognise the single most common testing configuration in Python repos.

Security notes

Standard dev-tool security posture; pytest executes user-authored test code with full developer privileges, so test files should be reviewed with the same scrutiny as production code. Plugin ecosystem is large and worth pinning by version.

Addresses failure modes

  • Incomplete execution — Agent claims completion while subtasks are skipped, stubbed, or silently dropped.
  • Test manipulation — Agent gaming its own evaluation — editing tests or test harness so failures disappear without fixing the underlying bug.
  • Logic error — Correct API use, incorrect logic.

Techniques

test-discoveryfixture-injectionparametrized-testingassertion-introspection

View raw YAML on GitHub →