Skip to main content

Troubleshooting

Common issues and how to resolve them.

Installation Issues​

Node.js Version Error​

Error: Bellwether requires Node.js 20 or later

Solution: Update Node.js:

# Check version
node --version

# Use nvm to install/switch
nvm install 20
nvm use 20

Permission Denied (Global Install)​

EACCES: permission denied

Solution: Fix npm permissions or use npx:

# Option 1: Use npx (no install needed)
npx @dotsetlabs/bellwether interview npx server

# Option 2: Fix npm permissions
sudo chown -R $(whoami) ~/.npm
npm install -g @dotsetlabs/bellwether

LLM Provider Issues​

API Key Not Found​

Error: Failed to initialize LLM client - API key not found

Solution: Set up your API key using one of these methods:

# Option 1: Interactive setup (recommended)
bellwether auth

# Option 2: Environment variable
export OPENAI_API_KEY=sk-xxx
# or
export ANTHROPIC_API_KEY=sk-ant-xxx

# Verify configuration
bellwether auth status

For persistent storage, bellwether auth stores your key in the system keychain (most secure) or in ~/.bellwether/.env.

Invalid API Key​

Error: 401 Unauthorized

Solution:

  1. Verify your API key is correct
  2. Check the key hasn't expired
  3. Ensure you have API access (not just ChatGPT access)

Rate Limiting​

Error: 429 Too Many Requests

Solution:

  • Wait and retry
  • Use --max-questions 1 to reduce API calls
  • Consider upgrading your API plan

Ollama Connection Refused​

Error: connect ECONNREFUSED 127.0.0.1:11434

Solution: Start Ollama:

# Start Ollama server
ollama serve

# Verify it's running
curl http://localhost:11434/api/tags

MCP Server Issues​

Connection Failed​

Error: Failed to connect to MCP server

Solutions:

  1. Verify the command works:

    # Test the server directly
    npx @modelcontextprotocol/server-filesystem /tmp

  2. Check for missing dependencies:

    npm install

  3. Increase timeout:

    bellwether interview --timeout 60000 npx server

Server Crashes​

Error: Server process exited unexpectedly

Solutions:

  1. Check server logs:

    bellwether interview --debug npx server

  2. Test server independently:

    npx server 2>&1

  3. Check for missing environment variables the server needs

Tool Not Found​

Error: Tool 'my_tool' not found

Solution: Verify available tools:

bellwether discover npx server

Interview Issues​

Timeout During Interview​

Error: Tool call timed out

Solution: Increase timeout:

bellwether interview --timeout 120000 npx server

Empty Results​

Warning: No tools discovered

Solutions:

  1. Check server supports MCP:

    bellwether discover npx server

  2. Verify server implementation returns tools in tools/list

Poor Quality Results​

Solutions:

  1. Use a better model:

    bellwether interview --model gpt-4o npx server

  2. Increase questions:

    bellwether interview --max-questions 5 npx server

  3. Use multiple personas:

    bellwether interview --persona technical_writer,security_tester npx server

Drift Detection Issues​

Baseline Not Found​

Error: Baseline file not found: ./bellwether-baseline.json

Solution: Create a baseline first:

bellwether interview --save-baseline npx server

Unexpected Drift​

If you're seeing drift you don't expect:

  1. Review the diff output to understand what changed

  2. Check if behavior actually changed vs. LLM interpretation variance

  3. Update baseline if changes are intentional:

    bellwether interview --save-baseline npx server
    git add bellwether-baseline.json
    git commit -m "Update baseline"

CI/CD Issues​

Exit Code 1 in CI​

Error: Behavioral drift detected

This is expected when drift is found. Options:

  1. Review and fix the drift
  2. Update the baseline if changes are intentional
  3. Remove --fail-on-drift to allow drift

Exit Code 2 in CI​

Error: Interview failed

Solutions:

  1. Check API key is set in CI secrets

  2. Enable debug logging:

    - run: |
        bellwether interview --debug --log-file debug.log npx server
    - uses: actions/upload-artifact@v4
      if: failure()
      with:
        name: debug-logs
        path: debug.log

  3. Test locally first:

    bellwether interview --ci npx server

Cloud Issues​

Login Failed​

Error: Authentication failed

Solutions:

  1. Try logging in again:

    bellwether login --logout
    bellwether login

  2. Check your account at bellwether.sh

Upload Failed​

Error: Failed to upload baseline

Solutions:

  1. Check you're logged in:

    bellwether login --status

  2. Verify project is linked:

    cat .bellwether.json

  3. Re-link if needed:

    bellwether link

Debug Mode​

For detailed troubleshooting, enable debug mode:

bellwether interview \
  --debug \
  --log-level debug \
  --log-file ./debug.log \
  npx server

This logs:

  • MCP protocol messages
  • LLM requests and responses
  • Tool call details
  • Timing information

Getting Help​

If these solutions don't help:

  1. Search existing issues: github.com/dotsetlabs/bellwether/issues

  2. Open a new issue with:

    • Bellwether version (bellwether --version)
    • Node.js version (node --version)
    • Command you ran
    • Full error message
    • Debug log if available

  3. Ask in Discussions: github.com/dotsetlabs/bellwether/discussions