summarization-ab (63f9db5025e8)
2026-06-01T21:01:17.501440+00:00 · model: claude-sonnet
Version A prompt
system:
You are a concise summarizer.
user:
Summarize the following in exactly 3 bullet points:
{{document}}
Version B prompt
system:
You are an expert editor writing for busy founders.
user:
Write a tight 3-bullet summary that a busy founder could skim.
Cover what the product does and its pricing.
{{document}}
Case: propeller-doc
Winner: A
improved ✗ ·
A 8.0
→ B 8.0
Both are faithful and useful, but Output B's 'Why it matters' bullet adds an editorializing framing ('not just likes') not in the source and slightly drifts from a strict summary, while Output A is cleanly faithful and concise; the improvement is marginal at best and arguably introduces unsourced framing.
Output A · $0.00154 · 3423ms
• Propeller is a Toronto-based company offering an AI-powered social media marketing tool designed exclusively for Shopify merchants. • The platform creates platform-specific social posts from store products and brand assets, suggests optimal posting times, and tracks sales attribution to individual posts. • Pricing begins at approximately $190/year, with a 14-day free trial available.
Output B · $0.00195 · 4855ms
• **What it does:** Propeller is an AI tool built exclusively for Shopify merchants that auto-generates platform-specific social posts using your existing products and brand assets, then recommends the best times to post. • **Why it matters:** It closes the loop with sales attribution, so you can see exactly which posts are driving revenue—not just likes. • **Cost:** Starts at ~$190/year with a 14-day free trial to test before committing.