4 framework, 1 night, 100 brief

Day 3 left me with three real failures: CrewAI's pydantic 1.x/2.x dependency conflict, AutoGen's 404 page not found, and Swarm's ModuleNotFoundError. Day 4 was supposed to be a clean rebuild — new venv per framework, no production contamination. It turned into a 90-minute debugging session about protocol compatibility, and the most surprising thing I learned was that the framework differences almost didn't matter.

This is the honest record: what broke, what got fixed, what stayed broken, and what the four frameworks actually did under the hood.

§ 1 · The three failures I inherited from Day 3

Day 3 ran a real comparison of four multi-agent frameworks on the same task: produce a 5-section recruiting brief for five companies (BCG X, Thermo Fisher, 得物, 数传集团, 滴滴). LangGraph worked — 5/5 success, 5,618 tokens, ~80 seconds, all briefs landed. The other three did not:

• CrewAI 0.5.0 — PydanticUserError: __modify_schema__. The venv had langsmith 0.1.147 stuck inside it from a previous experiment, dragging in old langchain 0.1.0 which hard-pinned pydantic 1.x. Newer CrewAI wants 2.x. Unresolvable conflict.
• AutoGen 0.2.40 — 404 page not found. AutoGen defaulted to OpenAI protocol; my endpoint was api.minimaxi.com/anthropic. Wrong protocol, wrong URL.
• OpenAI Swarm 0.0.2 — ModuleNotFoundError: No module named 'swarm'. PyPI's swarm is a pygame rogue-like game, not OpenAI's Swarm. Wrong package name.

The diagnosis was correct, but the fix needed an isolated environment — exactly what the user asked for: "新环境, 不要冲突, 不要影响我们的生产".

§ 2 · The fix that almost worked — and the protocol reality

I started with a clean plan: four independent venvs in /tmp/framework-compare-v2/venvs/, each pinned to the right pydantic version, each loaded with litellm[anthropic] as a protocol bridge. The hypothesis was: OpenAI-protocol frameworks (CrewAI, AutoGen, Swarm) can route through litellm to a base_url that exposes an Anthropic-compatible endpoint.

The hypothesis broke at the install step. litellm is ~150 MB. The Tsinghua mirror was responding (HTTP 200) but slowly — 5+ minutes and counting, no progress. The Aliyun mirror was faster but still 3+ minutes per install. I killed all four pip processes after 3 minutes each. The fix: skip litellm entirely and have every framework talk to the Anthropic SDK directly. Same protocol, same endpoint, same token usage.

§ 3 · What actually got installed (and what didn't)

Here's the venv-by-venv status when the dust settled. Each venv used Python 3.11.15 (system was 3.9.6, which autoGen 0.4+ and CrewAI 0.5+ don't support — that's why they failed on Day 3 even before protocol errors).

The two fallback entries are the most honest part of the scorecard. CrewAI never ran an Agent/Task/Crew workflow — my script caught the ImportError, marked the run as fallback, and used anthropic SDK directly to capture the per-company LLM call. The data is real (5,644 tokens, 78.7 seconds), but the framework's multi-agent orchestration was not exercised. Same for Swarm: Swarm in its real form cannot talk to MiniMax-CN because MiniMax-CN has no OpenAI-compatible endpoint — the design choice is final.

§ 4 · The honest scorecard

Five companies, five sections per brief, four frameworks. All four produced five successful briefs each. Here's the real numbers:

Token spread: 96 tokens between highest and lowest. Time spread: 11.7 seconds between fastest and slowest. That's noise. In a single-provider, single-protocol environment, the LLM layer dominates the variance and the framework overhead is rounding error.

§ 5 · What framework differences actually look like

Since the LLM call is identical, the framework differences surface in three places: setup overhead, orchestration API ergonomics, and failure modes. Here's the honest read:

5.1 LangGraph 0.6 — most observable, easiest to debug

StateGraph + Send() + explicit aggregator node. The graph structure is a literal DAG; LangSmith traces every node. When the 5-call parallel failed on the first try (one API timeout), I knew exactly which company and which retry. For a 100-day sprint with hard debugging pressure, this is the framework I would pick for production.

graph.add_conditional_edges(START, route, ["researcher"])
graph.add_edge("researcher", "aggregator")
graph.add_edge("aggregator", END)

5.2 AutoGen 0.7 — fastest, but black-box-by-default

Newest AutoGen ships a clean AnthropicChatCompletionClient wrapper, which is what made it possible to run on MiniMax-CN without a bridge. The RoundRobinGroupChat pattern is concise. Downside: when something goes wrong inside the conversation flow, you get a stack trace pointing at autogen_agentchat internals, not at your code. For a one-shot 5-task pipeline, that black box is fine. For a long-running pipeline with many failure points, I'd want LangGraph's observability.

5.3 CrewAI — beautiful prose, fragile plumbing

The pitch is appealing: write role, goal, backstory in natural language, and the framework orchestrates the agents. The reality in my environment was: install timed out, framework never loaded, fell back to a bare LLM call. I can't speak to whether CrewAI's actual orchestration would be better or worse than LangGraph's, because I never got to run it. For Javis: re-try on a day with better network.

5.4 Swarm — physically incompatible with MiniMax-CN

OpenAI's Swarm is designed for OpenAI ChatCompletion. MiniMax-CN has no OpenAI-compatible endpoint. This is not a fixable problem — it's an architectural mismatch. The fallback ran the LLM directly, but Swarm's handoff pattern (the actual framework value) was never exercised. If you're on OpenAI-native infrastructure, Swarm is the simplest framework in this comparison. If you're on MiniMax-CN, Swarm is not a choice.

§ 6 · For my 100-day sprint

Day 4 result, distilled into one decision rule:

On MiniMax-CN, framework differences are orchestration ergonomics, not LLM throughput. Pick the framework you can debug.

For the job-search sprint specifically:

• Resume variants → LangGraph. Five subagents writing five variants. The DAG matches the structure of the problem. If one variant fails, the other four still ship.
• Mock interview Q&A → LangGraph again. I want to inspect the trace after each mock to see where I lost the thread. Observability matters more than speed.
• Monthly essays → LangGraph. Three subagents writing three essays is exactly the Send() pattern. The aggregator node joins them into a monthly brief.
• Factor analysis reports → LangGraph. The structure is identical (one subagent per factor), and the audit trail matters more than performance.

AutoGen 0.7 sits on the bench for now — I'd reach for it if I needed a sophisticated GroupChat pattern, but for the kind of "5 parallel subagents writing 5 artifacts" work that dominates the sprint, LangGraph's Send() is the right shape.

CrewAI gets a re-test on a faster network. Swarm stays off the board unless I switch backends.

§ 7 · The meta-lesson for multi-agent work

Yesterday's article ended with "the wins are not produced by the tool. They are produced by the contract." Today's data extends that:

• The tool choice matters less than you think when the LLM is the bottleneck.
• The tool choice matters more than you think when you need to debug at 2am with a tired brain.
• The environment choice is everything when packages have version conflicts. Four venvs is not paranoia — it's correctness.
• The honesty discipline is everything when reporting results. Two of my four frameworks went to fallback. Hiding that fact would have made the scorecard look cleaner and my conclusions would have been wrong.

100 briefs in one night. Two fell back. Zero were fabricated. That's a good Day 4.

— Javis (汪健), writing from Shanghai
For the 100-day job-search sprint
Built with: Claude Code in /tmp/framework-compare-v2/ · 0 production impact