vet

commit a42bb58fad83ed8d2c6e1337656bad1fb74b1767
parent ce08a692a1c4d840f371c324cada7c7b25cbd864
Author: andrewlaack-collab <andrew.laack@imbue.com>
Date:   Fri,  6 Feb 2026 14:31:47 -0800

Updated default model, updated CI stuff, added ci instructions. (#22)

* Updated default model, updated CI stuff, added ci instructions.

* Updated readme

* Black

* Only on non-draft PRs
Diffstat:
M
.github/workflows/test.yml
 | 
20
++++++++++----------
M
README.md
 | 
58
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
M
skills/vet/SKILL.md
 | 
2
+-
M
vet/cli/models.py
 | 
2
+-

4 files changed, 70 insertions(+), 12 deletions(-)
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -8,6 +8,7 @@ on:
 
 jobs:
   test:
+    if: github.event.pull_request.draft == false
     runs-on: ubuntu-latest
 
     steps:
@@ -33,22 +34,21 @@ jobs:
 
       - name: Run vet
         if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository
-        continue-on-error: true
         env:
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           VET_GOAL: |
             ${{ github.event.pull_request.title }}
 
             Additional context (not necessarily part of the goal):
             ${{ github.event.pull_request.body }}
-        run: uv run vet "$VET_GOAL" --base-commit main --confidence-threshold 0.9 --max-workers 4 --quiet --model opus > vet-output.txt 2>&1
-
-      - name: Comment vet results
-        if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
-          if [ ! -s vet-output.txt ]; then
-            echo "Vet found no issues." > vet-output.txt
+          set +e
+          uv run vet "$VET_GOAL" --base-commit main --confidence-threshold 0.9 --max-workers 4 --quiet --model opus > "$RUNNER_TEMP/vet-output.txt" 2>&1
+          status=$?
+          if [ ! -s "$RUNNER_TEMP/vet-output.txt" ]; then
+            echo "Vet found no issues." > "$RUNNER_TEMP/vet-output.txt"
           fi
-          gh pr comment ${{ github.event.pull_request.number }} --body-file vet-output.txt
+          gh pr comment "${{ github.event.pull_request.number }}" --body-file "$RUNNER_TEMP/vet-output.txt"
+          if [ "$status" -eq 1 ]; then exit 0; fi
+          exit "$status"
diff --git a/README.md b/README.md
@@ -24,6 +24,64 @@ Compare against a base ref/commit:
 vet "Refactor storage layer" --base-commit main
 ```
 
+## GitHub PRs (Actions)
+
+Vet can run on pull requests.
+
+Create `.github/workflows/vet.yml`:
+
+```yaml
+name: Vet
+
+permissions:
+  contents: read
+  pull-requests: write
+  issues: write
+
+on:
+  pull_request:
+    types: [opened, edited, synchronize, reopened]
+
+jobs:
+  vet:
+    if: github.event.pull_request.draft == false
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: pip install git+https://github.com/imbue-ai/vet.git
+      - name: Run vet
+        if: github.event.pull_request.head.repo.full_name == github.repository
+        env:
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          VET_GOAL: |
+            ${{ github.event.pull_request.title }}
+
+            Additional context (not necessarily part of the goal):
+            ${{ github.event.pull_request.body }}
+        run: |
+          set +e
+          vet "$VET_GOAL" --quiet --base-commit "${{ github.event.pull_request.base.sha }}" > "$RUNNER_TEMP/vet-output.txt" 2>&1
+          status=$?
+          if [ ! -s "$RUNNER_TEMP/vet-output.txt" ]; then
+            echo "Vet found no issues." > "$RUNNER_TEMP/vet-output.txt"
+          fi
+          gh pr comment "${{ github.event.pull_request.number }}" --body-file "$RUNNER_TEMP/vet-output.txt"
+          if [ "$status" -eq 1 ]; then exit 0; fi
+          exit "$status"
+```
+
+NOTE: This will not fail in CI if Vet finds an issue. This will only add a comment to the PR.
+
+#### Environment variables
+
+- `ANTHROPIC_API_KEY` is required for the default model configuration.
+
 ## Using Vet with Coding Agents
 
 Vet ships as an [agent skill](https://agentskills.io) that coding agents like [OpenCode](https://opencode.ai) and [Codex](https://github.com/openai/codex) can discover and use automatically. When installed, agents will proactively run vet after code changes and include conversation history for better analysis.
diff --git a/skills/vet/SKILL.md b/skills/vet/SKILL.md
@@ -64,7 +64,7 @@ vet "goal"
 ## Common Options
 
 - `--base-commit REF`: Git ref for diff base (default: HEAD)
-- `--model MODEL`: LLM model to use (default: claude-4-5-sonnet)
+- `--model MODEL`: LLM model to use (default: claude-4-6-opus)
 - `--confidence-threshold N`: Minimum confidence 0.0-1.0 (default: 0.8)
 - `--output-format FORMAT`: Output as `text` or `json`
 - `--quiet`: Suppress progress output
diff --git a/vet/cli/models.py b/vet/cli/models.py
@@ -9,7 +9,7 @@ from vet.cli.config.loader import get_models_by_provider_from_config
 from vet.cli.config.loader import get_user_defined_model_ids
 from vet.cli.config.schema import ModelsConfig
 
-DEFAULT_MODEL_ID = AnthropicModelName.CLAUDE_4_5_SONNET_2025_09_29.value
+DEFAULT_MODEL_ID = AnthropicModelName.CLAUDE_4_6_OPUS.value
 
 
 def get_builtin_model_ids() -> set[str]: