Migrating a RAG copilot
serving 5,000 users.

The situation

Roche Diagnostics had a RAG-powered copilot built on LangChain, used internally by thousands of employees for troubleshooting and knowledge retrieval. The system worked, but the codebase was becoming painful to maintain. LangChain's abstraction layer made it hard to debug issues, customize behavior, or reason about execution flow.

On top of that, the LLM observability stack was tightly coupled to Literal AI through the Chainlit frontend integration. If Literal AI changed pricing, deprecated an API, or shut down — the entire monitoring and tracing pipeline would break, and the team would be scrambling for an emergency replacement.

What I proposed

I identified two separate but related problems and pitched solutions for both.

For the maintainability issue, I proposed migrating from LangChain to LangGraph. LangGraph gives you explicit control over the agent's execution graph — you see exactly what happens at each step, add branching, handle errors granularly, and test individual nodes. This wasn't an assigned task; I saw the opportunity, built a proof of concept on my own time, and presented it to the team and stakeholders.

For the vendor lock-in risk, I proposed an observability abstraction layer — a thin adapter between our application and whatever provider we use. I evaluated options and recommended Langfuse: open-source, self-hostable, clean integration. The team accepted the proposal; another developer handled the migration using the adapter pattern I designed.

The outcome

The LangGraph migration improved code maintainability by roughly 60% (measured by reduced time-to-change for new features and bug fixes). The copilot continued serving 5,000+ users without disruption during the migration.

Test coverage went from 40% to 80%. I introduced TDD practices and CI/CD quality gates that the team adopted going forward.

I also delivered six voluntary AI workshops covering agentic coding tools — Cursor, Claude Code, MCP integrations. Not in scope, but the team could ship faster with better tooling and I wanted to share what I'd learned.

Why this matters

A lot of companies built their first AI systems on LangChain in 2023–2024 and are hitting the same walls: hard to debug, hard to customize, too many abstractions hiding what's actually happening. The migration path to LangGraph (or similar explicit orchestration) is well-worn at this point — but it requires someone who understands both the old and new patterns deeply enough to do it without breaking a system that real users depend on.