AI Note Takers: A Production-Ready Technical Comparison

What actually differentiates an AI note taker

Every AI note taker on the market records a meeting and returns a summary. Feature lists converge fast, so they are a poor way to choose. The decisions that survive contact with production are architectural, and there are four of them:

• Capture — does a bot join the call as a visible participant, or does a desktop client capture system audio silently?
• Transcription — does audio travel to a cloud ASR service, or does a model run on the user's machine?
• Grounding — is the summary generated from one transcript in a single prompt, or retrieved across a searchable history?
• Action — does the output stop at notes, or does it write to the systems where the work actually happens?

Each of those choices sets a ceiling on accuracy, latency, cost, and what a security review will allow. The rest of this guide walks through them.

Platform comparison at a glance

A structural comparison of how the leading platforms approach capture, transcription, and data handling. Details change as vendors ship, so treat this as an architectural map rather than a feature scorecard.

Capture: meeting bots vs. system audio

The bot model

Otter.ai, Fathom, and Fireflies all default to a bot that joins the call over the platform's API. It is the only approach that works when the note taker must attend a meeting the user is not in, and it gets clean, per-participant streams that make diarization far easier.

The engineering cost is real. A bot fleet is a distributed system: joins fail, waiting rooms block entry, hosts deny admission, tokens expire, and calls run long. Building Sally Web taught me to treat every join as a queue-backed job with checkpoints and idempotent retries — a dropped connection resumes the session instead of losing the meeting. Anything less and your reliability is capped by the flakiest conferencing API you integrate.

The desktop model

Granola and Sally Desktop capture system audio locally instead. Nothing appears in the participant list, which matters for client calls, interviews, and any conversation where an unexplained bot changes the tone of the room. It also works on platforms with no bot API at all, including in-person conversations.

The tradeoff is that you inherit OS-level complexity: audio loopback permissions differ across macOS and Windows, and capture must survive sleep, device switching, and headphone changes mid-call. Sally Desktop handles this in Rust via Tauri 2 precisely because that layer needs native control and a small footprint.

Transcription: cloud STT vs. on-device Whisper

Cloud ASR remains the accuracy leader on hard audio — heavy accents, crosstalk, poor microphones — and it scales horizontally without touching the client. Every bot-based platform uses it, because the audio is already server-side.

On-device Whisper changes the tradeoff profile completely:

• Privacy: raw audio never leaves the machine, which removes an entire category of vendor-risk questions.
• Cost: no per-minute transcription billing, so heavy-usage teams get a flat cost curve.
• Offline: transcription works on a plane or behind a restrictive network.
• Constraint: quality and speed depend on the user's hardware and the model size you can afford to load.

In practice the honest answer is a hybrid: run a quantized Whisper model locally for the default path, and let the user opt into cloud transcription for difficult audio or low-powered machines. That is the split Sally Desktop and Sally Web occupy — same product surface, two very different data paths, chosen per customer's threat model.

Summarization and retrieval quality

Summary quality is where most comparisons stop at "it writes good notes." Technically, there are two distinct problems.

Single-meeting summarization is largely solved. A modern long-context model handles an hour-long transcript in one pass; the differentiators are structured extraction — decisions, owners, due dates as typed fields rather than prose — and speaker attribution good enough that action items land on the right person.

Cross-meeting recall is not solved, and it is what separates a note taker from meeting intelligence. "What did we promise this client in Q2?" spans dozens of transcripts and needs a real retrieval pipeline: speaker- and topic-aware chunking, embeddings in a vector store, hybrid keyword plus semantic search, reranking, and strict tenant scoping at query time. Sally Web runs this on Postgres with pgvector; the full architecture is in my guide on building production-grade RAG systems.

A useful evaluation trick: ask each platform a question whose answer lives in three different meetings. Tools that only summarize will confidently answer from the most recent one.

Privacy, residency, and compliance

This is usually the deciding factor in enterprise procurement, and it maps directly to architecture. Cloud-first platforms store recordings, transcripts, and embeddings on vendor infrastructure — fine for most teams, blocking for legal, healthcare, and regulated finance. The questions that matter are retention windows, whether customer audio is used for model training, region of storage, and whether a per-tenant deletion actually removes derived embeddings and not just the transcript row.

On-device transcription sidesteps most of it: with Sally Desktop, audio and transcripts stay local unless the user explicitly syncs. For multi-tenant SaaS the equivalent discipline is enforcing tenant isolation in the data layer and scoping every retrieval query by tenant, so a filter bug cannot leak another customer's meeting into an answer.

Integrations and downstream actions

Notes nobody reads are waste. The measurable difference between platforms is whether output reaches the CRM, ticket tracker, and calendar where the follow-up lives. Fathom leans hard into sales-CRM push; Fireflies covers a wide integration surface; Otter emphasizes shared workspaces and search.

Architecturally, the step beyond a webhook is a bounded agent loop: typed, permission-checked tools for creating tasks, syncing CRM records, and scheduling follow-ups, with every argument validated server-side before execution. That is how Sally Web turns "who owns this?" into an actual record in the customer's system — the orchestration patterns are covered in my guide on agentic AI workflows.

How the Sally suite is architected

Sally exists as two products because one architecture cannot serve both buyers.

Sally Web

A multi-tenant B2B SaaS platform. Queue-backed bot orchestration joins calls with checkpointed, resumable jobs. Transcripts are chunked and embedded into Postgres with pgvector, retrieved through hybrid search scoped per tenant, and served behind an intent router that answers metadata questions straight from Postgres instead of paying for a vector lookup. Action requests run through typed tools across 16+ integrations.

Sally Desktop

A privacy-first native client built on Tauri 2 — Rust core, Next.js interface — that captures system audio without joining the meeting and transcribes on-device with Whisper. No bot in the participant list, no audio leaving the machine, and a flat cost profile regardless of meeting volume.

Same problem domain, opposite constraints: Sally Web optimizes for organizational recall across a shared history, Sally Desktop optimizes for data never leaving the endpoint.

Choosing the right architecture

• Meetings you do not attend — you need a bot-based platform; there is no local-capture substitute.
• Confidential or client-facing calls — a desktop capture client with on-device transcription avoids both the visible bot and the vendor data path.
• Questions spanning months of meetings — you need retrieval over a searchable history, not per-meeting summaries.
• Sales and support workflows — weight CRM depth and action reliability over transcript polish.
• Regulated environments — start with retention, training-data, and residency terms; they eliminate most of the shortlist before features matter.

The short version: pick the capture model your meetings allow, pick the transcription location your compliance posture requires, and only then compare summaries.