vet

Mirror of Vet, an AI code review tool
git clone git://git.laack.co/vet.git
Log | Files | Refs | README | LICENSE

DEVELOPMENT.md (6559B)

      1 # Development
      2 
      3 For general usage, installation, and configuration, see the [README](README.md).
      4 
      5 ## Dev Setup
      6 
      7 ### On your host machine
      8 
      9 Ensure you have [uv](https://docs.astral.sh/uv/getting-started/installation/) installed, and that you have the correct env variables set to run Vet (Vet defaults to Anthropic models so this means you should have your ANTHROPIC_API_KEY set).
     10 
     11 Then run:
     12 
     13 ```bash
     14 uv run vet
     15 ```
     16 
     17 ### Containerized
     18 
     19 You can use the `Containerfile` in `dev/` at the repo root to create a container that suffices to run Vet for development purposes.
     20 
     21 #### Setup
     22 
     23 Create a `.env` file at the repo root with your API keys. The recommended keys are `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, and `CODEX_API_KEY`.
     24 
     25 To include Claude Code in the image add:
     26 
     27 ```
     28 I_CHOOSE_CONVENIENCE_OVER_FREEDOM=true
     29 ```
     30 
     31 Without this Claude Code will not be installed in the image.
     32 
     33 #### Running Vet in a Container
     34 
     35 ```bash
     36 ./dev/vet.sh --list-models
     37 ./dev/vet.sh "check for bugs" --base-commit main
     38 ./dev/vet.sh --base-commit main --agentic --agent-harness codex
     39 ./dev/vet.sh --base-commit main --agentic --agent-harness claude  # requires I_CHOOSE_CONVENIENCE_OVER_FREEDOM=true
     40 ./dev/vet.sh --base-commit main --agentic --agent-harness opencode
     41 ```
     42 
     43 The image is built automatically on each run. This process should be fast due to layer caching.
     44 
     45 #### Interactive Development
     46 
     47 ```bash
     48 ./dev/run.sh
     49 ```
     50 
     51 Starts an interactive shell in the container. The repo is bind-mounted at `/app`.
     52 
     53 ## Formatting Hooks
     54 
     55 Install pre-commit hooks once per clone:
     56 
     57 ```bash
     58 uvx pre-commit install
     59 ```
     60 
     61 Run formatting hooks manually across the repo:
     62 
     63 ```bash
     64 uvx pre-commit run --all-files
     65 ```
     66 
     67 After installation, `isort` and `black` run automatically on staged Python files before each commit.
     68 
     69 ## Running Tests
     70 
     71 ### Unit tests
     72 
     73 All unit tests are run with:
     74 
     75 ```bash
     76 uv run pytest
     77 ```
     78 
     79 This command should be preserved the sole way to run unit tests.
     80 
     81 ## Concepts
     82 
     83 ### Issue identifiers
     84 
     85 Issue identifiers are pieces of logic capable of finding issues in code. We foresee two basic kinds of those:
     86 
     87 1. File-checking ones.
     88     - To check for "objective" issues in existing files.
     89 2. Commit-checking ones.
     90     - To check for the quality of a single commit.
     91     - "Assuming that we can treat the commit message as a requirement, how well does the commit implement it?"
     92 
     93 By default, `vet` runs all the registered issue identifiers and outputs all the found issues on the standard output in JSON format.
     94 
     95 #### Adding new Issue Identifiers
     96 
     97 If you want to add a new issue identifier, you need to:
     98 
     99 1. Implement the `IssueIdentifier` protocol from `vet.imbue_tools.repo_utils.data_types`.
    100 2. Register the new issue identifier by adding it to `IDENTIFIERS` in `vet.issue_identifiers.registry`.
    101 
    102 Based on your needs, instead of the above, you can also extend one of the existing batched zero-shot issue identifiers:
    103     - `vet/issue_identifiers/batched_commit_check.py`
    104       (for commit checking)
    105 In that case you would simply expand the rubric in the prompt. That is actually the preferred way to catch issues at the moment due to efficiency.
    106 Refer to the source code for more details.
    107 
    108 ### Model registry
    109 
    110 The `registry/models.json` file contains model definitions distributed via the `--update-models` CLI option. See [`registry/CONTRIBUTING.md`](registry/CONTRIBUTING.md) for expectations about what models should be added to the registry.
    111 
    112 ## CI / CD
    113 
    114 ### GitHub Actions naming conventions
    115 
    116 Workflows follow a consistent naming scheme across three layers:
    117 
    118 - **File name**: `<verb>-<target>.yml` (e.g. `test-unit.yml`)
    119 - **Display name** (`name:`): `<Verb> / <Target>` (e.g. `Test / Unit`)
    120 - **Job name**: short target identifier (e.g. `unit`)
    121 
    122 The `/` in display names creates visual grouping in the GitHub Actions UI. Group related workflows under a shared prefix (e.g. `Test /`, `Publish /`). Standalone workflows (e.g. `Vet`) don't need a prefix.
    123 
    124 Current workflows:
    125 
    126 - `test-unit.yml` (`Test / Unit`, job: `unit`) — pytest suite (lint + unit tests)
    127 - `test-pkgbuild.yml` (`Test / PKGBUILD`, job: `pkgbuild`) — Arch Linux package build + smoke test
    128 - `vet.yml` (`Vet`, job: `vet`) — Self-review via vet on PRs (uses the reusable action via `uses: ./`)
    129 - `vet-agentic.yml` (`Vet (Agentic)`, job: `vet`) — Agentic self-review via vet on PRs (uses the reusable action via `uses: ./`)
    130 - `publish-pypi.yml` (`Publish / PyPI`, job: `pypi`) — Build and publish to PyPI on tag push
    131 - `publish-github-release.yml` (`Publish / GitHub Release`, job: `github-release`) — Create a GitHub Release on tag push
    132 
    133 ### Continuous Deployment
    134 
    135 Vet is published to PyPI via the `publish-pypi.yml` GitHub Actions workflow. Deployment is triggered by pushing a git tag that starts with `v` (e.g. `v0.2.0`).
    136 
    137 ### Releasing a new version
    138 
    139 1. Create and checkout a branch to bump the version, using the naming convention `{name}/v{version}` (e.g. `john/v0.2.0`)
    140 2. Update the version in `pyproject.toml`
    141 3. Update `pkgver` in `pkg/arch/PKGBUILD`
    142 4. Commit and push the changes
    143 5. Tag the commit and push the tag:
    144    ```bash
    145    git tag v0.2.0 -m "v0.2.0: Updated XYZ"
    146    git push origin v0.2.0
    147    ```
    148 6. Create a PR for the new branch
    149 7. The `Publish / PyPI` workflow will automatically build and publish the package
    150 8. Merge PR into main.
    151 
    152 ## Development Notes
    153 
    154 ### Logging
    155 
    156 When creating a new entry point into vet, you must call `configure_logging(verbose: int, log_file: Path | None)` from `vet.cli.main`.
    157 
    158 User-facing status messages (top-level lifecycle, warnings visible to the user) use `print(..., file=sys.stderr)` directly — not loguru. Loguru is for internal diagnostics only.
    159 
    160 Log level heuristics:
    161 
    162 - **TRACE** - API payloads, token counts, dollar costs, agent subprocess messages.
    163 - **DEBUG** - Everything internal: API exceptions before re-raise, retries, fallbacks, identifier selection, history loading, context assembly. All LLM provider exception handlers must log at DEBUG before raising (see `_openai_exception_manager` for the pattern).
    164 - **WARNING** - Degraded conditions: LLM content blocked/flagged, unrecognized config values, malformed user data. Note: spend limit warnings also `print()` directly to stderr so they are always visible to the user.
    165 - **ERROR** - Failures that prevent producing results. Use `log_exception()` from `vet.imbue_core.async_monkey_patches` for tracebacks.
    166 
    167 ### README links
    168 
    169 The README is rendered on PyPI which does not resolve relative links that otherwise work on GitHub. Always use full URLs when linking to resources from the README.