Troubleshooting

Engine Not Reachable

Symptoms:

• CLI errors connecting to http://127.0.0.1:4041

Checks:

• engine running with uvicorn
• engine_addr config value
• local firewall/network constraints

Missing API Key

CLI run fails if provider key is missing.

Set one of:

• ANTHROPIC_API_KEY
• OPENAI_API_KEY
• GOOGLE_API_KEY

No Model Specified

validtr run requires --model — there is no default model. If you omit it the CLI exits with "no model specified — pass --model".

Model Not Found

A run failing with a model-not-found error (HTTP 400 from the engine) means the model id is misspelled or retired by the provider. Use a current model id — e.g. for Anthropic, claude-sonnet-4-6 or claude-opus-4-8. (Older dated ids such as claude-sonnet-4-20250514 have been retired.)

Docker Errors

Potential causes:

• Docker daemon not running
• insufficient permissions to Docker socket
• image build failures for base/per-run images

Empty or Weak Recommendations

Potential causes:

• no TAVILY_API_KEY (web search skipped)
• upstream MCP/skills registry fetch failures
• provider JSON formatting failures

Test Container Failures

Look for:

• malformed generated pytest
• test timeouts
• missing expected output files in /workspace/output

Runner output is included in TestSuiteResult.runner_output.