Skip to main content

What Bellwether Is (and Isn't)

Understanding where Bellwether fits in your development workflow.

Bellwether IS:

  • Drift detection system first: Compares baseline behavior and schemas to catch unexpected changes between versions
  • Core CI workflow: init + check + baseline for deterministic deployment gating
  • Behavioral documentation tool (optional layer): explore can generate deeper behavior docs when needed
  • Multi-perspective testing: 4 personas (Technical Writer, Security, QA, Novice) surface different types of issues
  • CI/CD integration: Designed to run in pipelines with exit codes for deployment gating
  • Security hygiene checker: Catches common issues like path traversal, injection patterns, and info disclosure

Bellwether IS NOT:

  • A replacement for unit tests: Use both. Unit tests verify expected behavior, Bellwether discovers unexpected behavior.
  • A security certification: Bellwether's security tests catch common issues, not professional security audits
  • A substitute for code review: Bellwether tests behavior, not code quality
  • Enterprise software: Built for individuals and small teams, not large organizations with SSO/SAML requirements

Bellwether vs. Traditional Testing

ApproachWhat it catchesWhat it misses
Unit testsRegressions in expected behaviorBehaviors you didn't think to test
Integration testsSystem-level failuresEdge cases in tool interactions
Manual testingIssues you look forIssues you don't know to look for
BellwetherUnexpected behaviors across 4 personas(Use with above for complete coverage)

Key insight: Unit tests verify YOUR expectations. Bellwether discovers UNEXPECTED behaviors.

Think of the difference:

  • Unit test: "Does get_weather('NYC') return weather data?"
  • Bellwether: "What happens when someone calls get_weather with a SQL injection string?"

They're complementary. Use both.

When to Use Bellwether

ScenarioBellwether helps?Notes
Building an MCP serverYesDocument behavior as you develop
Adopting a third-party MCP serverYesVerify behavior before trusting
CI/CD pipeline gatingYesUse Bellwether exit codes for deployment gates
Security compliance auditsPartialGood first step, not sufficient alone
Enterprise-wide rolloutNoNo SSO, limited team features

Complementary Tools

Bellwether works best alongside:

  • Unit testing frameworks (Jest, Vitest, pytest): Test expected behavior
  • Security scanners (Snyk, Dependabot): Dependency vulnerabilities
  • Professional security audits: For high-sensitivity systems
  • API documentation tools (OpenAPI): Schema documentation

Sustainability & Business Model

Bellwether is built by Dotset Labs, a bootstrapped software company.

Bellwether is fully open source and sustainable:

  1. MIT Licensed: The entire tool is open source. If the project is ever abandoned, the code is yours to fork and maintain.

  2. Community-Driven: Contributions welcome. The goal is community-maintained infrastructure, not a one-person dependency.

  3. No Lock-in: Everything works locally. Store baselines in git, run checks in CI, no accounts required.