README.md (9637B)
1 <p align="center"> 2 <a href="https://github.com/imbue-ai/vet"> 3 <img alt="Vet: Verify Everything" src="https://raw.githubusercontent.com/imbue-ai/vet/main/images/Vet.svg" width="21%"> 4 </a> 5 </p> 6 7 <p align="center"> 8 <a href="https://pypi.python.org/pypi/verify-everything/"><img src="https://img.shields.io/pypi/v/verify-everything.svg" alt="PyPi"></a> 9 <a href="https://www.gnu.org/licenses/agpl-3.0"><img src="https://img.shields.io/badge/License-AGPL_v3-blue.svg" alt="License: AGPL v3"></a> 10 <img src="https://github.com/imbue-ai/vet/actions/workflows/test-unit.yml/badge.svg" alt="Build Status"> 11 <a href="https://discord.gg/sBAVvHPUTE"><img src="https://img.shields.io/badge/Discord-Join%20us-5865F2?logo=discord&logoColor=white" alt="Discord"></a> 12 </p> 13 14 <p align="center">Vet is a standalone verification tool for <b>code changes</b> and <b>coding agent behavior</b>.</p> 15 16 ## Why Vet 17 18 - **Reviews intent and code**: checks agent conversations for goal adherence and code changes for correctness. 19 - **Runs anywhere**: from the terminal, as an agent skill, or in CI. 20 - **Bring-your-own-model**: works with any provider using your own API keys. 21 - **Works with existing subscriptions**: supports Anthropic and OpenAI subscriptions using [`--agentic`](#usage). 22 - **Free and open source**: no account, fees, or data collection. Requests go directly to your inference provider. Licensed under the AGPL-3.0. 23 24 <p align="center"> 25 <a href="https://github.com/imbue-ai/vet"> 26 <img alt="Vet: Verify Everything" src="https://raw.githubusercontent.com/imbue-ai/vet/main/images/vet_example.png" width="100%"> 27 </a> 28 </p> 29 30 ## Using Vet with Coding Agents 31 32 Vet includes an agent skill. When installed, agents will proactively run vet after code changes to find issues with the new code and mismatches between the user's request and the agent's actions. 33 34 ### Install the skill 35 36 ```bash 37 curl -fsSL https://raw.githubusercontent.com/imbue-ai/vet/main/install-skill.sh | bash 38 ``` 39 40 You will be prompted to choose between: 41 42 - **Project level**: installs into `.agents/skills/vet/`, `.opencode/skills/vet/`, `.claude/skills/vet/`, and `.codex/skills/vet/` at the repo root (run from your repo directory) 43 - **User level**: installs into `~/.agents/`, `~/.opencode/`, `~/.claude/`, and `~/.codex/` skill directories, discovered globally by all agents 44 45 ### Demo 46 47  48 49 <details> 50 <summary>Manual installation</summary> 51 52 #### Project Level 53 54 From the root of your git repo: 55 56 ```bash 57 for dir in .agents .opencode .claude .codex; do 58 mkdir -p "$dir/skills/vet/scripts" 59 for file in SKILL.md scripts/export_opencode_session.py scripts/export_codex_session.py scripts/export_claude_code_session.py; do 60 curl -fsSL "https://raw.githubusercontent.com/imbue-ai/vet/main/skills/vet/$file" \ 61 -o "$dir/skills/vet/$file" 62 done 63 done 64 ``` 65 66 #### User Level 67 68 ```bash 69 for dir in ~/.agents ~/.opencode ~/.claude ~/.codex; do 70 mkdir -p "$dir/skills/vet/scripts" 71 for file in SKILL.md scripts/export_opencode_session.py scripts/export_codex_session.py scripts/export_claude_code_session.py; do 72 curl -fsSL "https://raw.githubusercontent.com/imbue-ai/vet/main/skills/vet/$file" \ 73 -o "$dir/skills/vet/$file" 74 done 75 done 76 ``` 77 78 </details> 79 80 ### Security note 81 82 The `--history-loader` option executes the specified shell command as the current user to load the conversation history. It is important to review history loader commands and shared config presets before use. 83 84 ## Install the CLI 85 86 ```bash 87 pip install verify-everything 88 ``` 89 90 Or with [pipx](https://pipx.pypa.io/): 91 92 ```bash 93 pipx install verify-everything 94 ``` 95 96 Or with [uv](https://docs.astral.sh/uv/): 97 98 ```bash 99 uv tool install verify-everything 100 ``` 101 102 ### Usage 103 104 Run Vet in the current repo: 105 106 ```bash 107 vet "Implement X without breaking Y" 108 ``` 109 110 Compare against a base ref/commit: 111 112 ```bash 113 vet "Refactor storage layer" --base-commit main 114 ``` 115 116 Use Claude Code, Codex, or OpenCode instead of LLM APIs (`--agent-harness`: `claude`, `codex`, `opencode`): 117 118 ```bash 119 vet "Implement X without breaking Y" --agentic --agent-harness claude 120 ``` 121 122 ## GitHub PRs (Actions) 123 124 Vet reviews pull requests using a reusable GitHub Action. 125 126 Create `.github/workflows/vet.yml`: 127 128 ```yaml 129 name: Vet 130 131 permissions: 132 contents: read 133 pull-requests: write 134 135 on: 136 pull_request: 137 types: [opened, edited, synchronize, reopened] 138 139 jobs: 140 vet: 141 if: github.event.pull_request.draft == false 142 runs-on: ubuntu-latest 143 env: 144 ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} 145 steps: 146 - uses: actions/checkout@v4 147 with: 148 ref: ${{ github.event.pull_request.head.sha }} 149 fetch-depth: 0 150 - uses: imbue-ai/vet@main 151 with: 152 agentic: false 153 ``` 154 155 The action handles Python setup, vet installation, merge base computation, and posting the review to the PR. `ANTHROPIC_API_KEY` must be set as a repository secret when using Anthropic models (the default). See [`action.yml`](https://github.com/imbue-ai/vet/blob/main/action.yml) for all available inputs. 156 157 ## How it works 158 159 Vet snapshots the repo and diff, optionally adds a goal and agent conversation, runs LLM checks, then filters/deduplicates findings into a final list of issues. 160 161  162 163 ## Output & exit codes 164 165 - Exit code `0`: no issues found 166 - Exit code `1`: unexpected runtime error 167 - Exit code `2`: invalid usage/configuration error 168 - Exit code `10`: issues found 169 170 Output formats: 171 - `text` 172 - `json` 173 - `github` 174 175 ## Configuration 176 177 ### Model configuration 178 179 Vet supports custom model definitions using OpenAI-compatible endpoints via JSON config files searched in: 180 181 - `$XDG_CONFIG_HOME/vet/models.json` (or `~/.config/vet/models.json`) 182 - `.vet/models.json` at your repo root 183 184 #### Example `models.json` 185 186 ```json 187 { 188 "providers": { 189 "openrouter": { 190 "name": "OpenRouter", 191 "api_type": "openai_compatible", 192 "base_url": "https://openrouter.ai/api/v1", 193 "api_key_env": "OPENROUTER_API_KEY", 194 "models": { 195 "gpt-5.2": { 196 "model_id": "openai/gpt-5.2", 197 "context_window": 400000, 198 "max_output_tokens": 128000, 199 "supports_temperature": true 200 }, 201 "kimi-k2": { 202 "model_id": "moonshotai/kimi-k2", 203 "context_window": 131072, 204 "max_output_tokens": 32768, 205 "supports_temperature": true 206 } 207 } 208 } 209 } 210 } 211 ``` 212 213 Then: 214 215 ```bash 216 vet "Harden error handling" --model gpt-5.2 217 ``` 218 219 ### Model registry 220 221 Vet maintains a remote model registry with community-contributed model definitions. To fetch the latest definitions without upgrading vet: 222 223 ```bash 224 vet --update-models 225 ``` 226 227 This downloads model definitions from the [registry](https://github.com/imbue-ai/vet/blob/main/registry/models.json) and caches them locally at `~/.cache/vet/remote_models.json`. Once cached, registry models appear in `vet --list-models` and can be used with `--model` like any other model. 228 229 Model resolution priority (highest to lowest): 230 1. User config (`.vet/models.json` or `~/.config/vet/models.json`) 231 2. Builtin models (Anthropic, OpenAI, Gemini) 232 3. Registry models (cached via `--update-models`) 233 234 See [`registry/CONTRIBUTING.md`](https://github.com/imbue-ai/vet/blob/main/registry/CONTRIBUTING.md) for information about contributing model definitions to the registry. 235 236 ### Configuration profiles (TOML) 237 238 Vet supports named profiles so teams can standardize CI usage without long CLI invocations. 239 240 Profiles set defaults like model choice, enabled issue codes, output format, and thresholds. 241 242 See [the example](https://github.com/imbue-ai/vet/blob/main/.vet/configs.toml) in this project. 243 244 ### Custom issue guides 245 246 You can customize the guide text for the issue codes via `guides.toml`. Guide files are loaded from: 247 248 - `$XDG_CONFIG_HOME/vet/guides.toml` (or `~/.config/vet/guides.toml`) 249 - `.vet/guides.toml` at your repo root 250 251 #### Example `guides.toml` 252 253 ```toml 254 [logic_error] 255 suffix = """ 256 - Check for integer overflow in arithmetic operations 257 """ 258 259 [insecure_code] 260 replace = """ 261 - Check for SQL injection: flag any string concatenation or f-string formatting used to build SQL queries rather than parameterized queries 262 - Check for XSS: flag user-supplied data rendered into HTML templates without proper escaping or sanitization 263 - Check for path traversal: flag file operations where user input flows into file paths without validation against directory traversal (e.g. ../) 264 - Check for insecure cryptography: flag use of deprecated or weak algorithms (e.g. MD5, SHA1 for security purposes, DES, RC4) 265 - Check for hardcoded credentials: flag passwords, API keys, or tokens embedded directly in source code 266 """ 267 ``` 268 269 Section keys must be valid issue codes (`vet --list-issue-codes`). Each section supports three optional fields: `prefix` (prepends to built-in guide), `suffix` (appends to built-in guide), and `replace` (fully replaces the built-in guide). `prefix` and `suffix` can be used together, but `replace` is mutually exclusive with the other two. Guide text should be formatted as a list. 270 271 ## Community 272 273 New to Vet? [Read the launch post](https://imbue.com/product/vet/) for an intro and 2-minute demo. 274 275 Join the [Imbue Discord](https://discord.gg/sBAVvHPUTE) for discussion, questions, and support. For bug reports and feature requests, please use [GitHub Issues](https://github.com/imbue-ai/vet/issues). 276 277 ## License 278 279 This project is licensed under the [GNU Affero General Public License v3.0 (AGPL-3.0-only)](https://github.com/imbue-ai/vet/blob/main/LICENSE).