chat-rs: A multi-provider LLM framework for Rust enabling type-safe clients with tool calling and structured outputs

For developers building AI agents in Rust, the primary friction point has always been the fragmentation of provider APIs. chat-rs addresses this by providing a unified abstraction layer that doesn't just wrap APIs, but leverages Rust's type system to enforce valid configurations via the type-state pattern. The ability to swap a Gemini client for a Claude client with a single line of code—without rewriting the core logic—is a significant productivity win for teams hedging their bets on specific models. Technically, the framework stands out in its handling of the 'agentic loop.' While many SDKs leave tool-call orchestration to the developer, chat-rs automates the loop and introduces a sophisticated 'Human in the Loop' mechanism. By allowing the stream to pause for manual approval on sensitive tools, it provides a practical path for deploying autonomous agents in production environments where blind execution is a liability. From a systems perspective, the transport abstraction is a thoughtful touch. By decoupling the provider logic from the networking layer, the library supports everything from standard reqwest HTTP calls to WebSockets and custom tower/hyper implementations. The addition of a Router with built-in circuit breakers ensures that application reliability isn't tied to the uptime of a single LLM vendor, providing automatic failover and recovery timeouts. However, the dependency on specific macros like #[tool] and the requirement for newer Rust editions (1.94+) may be a hurdle for some legacy projects. The project is clearly aimed at the 'builder'—those who want granular control over their networking stack and compile-time guarantees rather than a high-level, opinionated wrapper. It is a solid choice for engineers building resilient, multi-model AI infrastructure.