Skip to content
UI Kit
Accent color

Compare outputs

Comparing AI outputs in four forms. A blind vote between two answers that reveals the models afterwards, a word diff between two versions, several models side by side with time, length and cost, and an evaluation table.

One of 4 components in AI answers

4 variants

Variants

Blind vote

Two answers labeled A and B, four choices to vote, then the model names flip in and the chosen card carries a ring.

Open in Studio

Text diff

Two versions compared word by word, added words fill in green, removed ones are struck through. Inline or side by side from tablet, changes only on request.

Open in Studio

Models side by side

The same prompt answered by several models, each with time, length and cost. Mark the best, copy any. Phones swipe with dots.

Open in Studio

Evaluation table

Test cases against models with pass or fail per cell, an average per model with bars and a filter for failing cases. Phones show one model at a time.

Open in Studio

Details

What Compare outputs brings along, read from the registry and the source.

Category
AI answers
Area
App
Variants
4
Version
1.0.0 · since 1.1.0
File
components/ui/compare-outputs.tsx
Brings along
utilskiticonrevealmotion
npm packages
None
Languages
English and German, 31 texts of its own

Props

The CompareOutputsProps interface from components/ui/compare-outputs.tsx, with the comments from the code.

  • variantoptional
    CompareOutputsVariant

    blind-votetext-diffmodel-grideval-table

  • titleoptional
    string
  • promptoptional
    string

    The prompt both outputs answer

  • outputsoptional
    CompareOutput[]
  • beforeoptional
    { label: string; text: string }

    text-diff: the earlier and the later version

  • afteroptional
    { label: string; text: string }
  • modelsoptional
    { id: string; name: string }[]

    eval-table

  • casesoptional
    EvalCase[]
  • passAtoptional
    number

    eval-table: score from which a case passes, 0 to 1

  • onVoteoptional
    (choice: string) => void

Built-in texts

Compare outputs writes these texts itself. The language follows the KitProvider, the labels prop replaces single ones.

Show all 31 texts
promptEnglish PromptGerman Auftrag
answerEnglish Answer {n}German Antwort {n}
whichEnglish Which answer is better?German Welche Antwort ist besser?
betterEnglish {n} is betterGerman {n} ist besser
tieEnglish Both are goodGerman Beide gut
badEnglish Both are poorGerman Beide schwach
revealedEnglish Thanks. Here is which model wrote what.German Danke. So verteilen sich die Modelle.
againEnglish Vote againGerman Neu abstimmen
blindNoteEnglish Models stay hidden until you vote, so the name does not sway you.German Die Modelle bleiben verborgen, bis du abstimmst, damit der Name nicht beeinflusst.
youPickedEnglish Your pickGerman Deine Wahl
changesEnglish ChangesGerman Änderungen
addedEnglish {n} addedGerman {n} neu
removedEnglish {n} removedGerman {n} entfernt
onlyChangesEnglish Changes onlyGerman Nur Änderungen
inlineEnglish InlineGerman Im Text
sideBySideEnglish Side by sideGerman Nebeneinander
timeEnglish TimeGerman Dauer
tokensEnglish LengthGerman Länge
costEnglish CostGerman Kosten
pinEnglish Mark as bestGerman Als beste markieren
pinnedEnglish Best answerGerman Beste Antwort
copyEnglish CopyGerman Kopieren
copiedEnglish CopiedGerman Kopiert
caseEnglish CaseGerman Fall
casesEnglish {n} casesGerman {n} Fälle
passedEnglish {n} of {total} passedGerman {n} von {total} bestanden
failingEnglish Show failing onlyGerman Nur nicht bestandene
passEnglish PassGerman Bestanden
failEnglish FailGerman Nicht bestanden
scoreNoteEnglish Scores come from an automatic check. Read the failing answers before you decide.German Die Werte kommen aus einer automatischen Prüfung. Lies die schwachen Antworten, bevor du entscheidest.
modelEnglish ModelGerman Modell

Accessibility

Read from the source of the component.

  • Compare outputs handles the keys Arrow left, Arrow right and Space itself.
  • Sets aria-current, aria-label, aria-live, aria-pressed and aria-selected, so screen readers announce purpose and state.
  • Uses the roles group, tab, tablist and tabpanel.
  • Announces changes through a live region without moving focus.
  • Every field has a label, visible or for screen readers.
  • Adds text for screen readers where icons and graphics stand alone.
  • The server renders the starting state in full, JavaScript only adds the interaction.

Motion

  • Rises in briefly the first time it comes into view (kit-rise, useInView).
  • Brings animations of its own, calmly timed and without wobble.
  • With reduced motion (prefers-reduced-motion) it stays still.

Source

The file aiw add puts into your project. From then on it is yours.

Show the file, 650 lines
components/ui/compare-outputs.tsx
'use client';

import { Fragment, useMemo, useRef, useState } from 'react';
import { cn, prefersReducedMotion, round, useLocale, useTexts } from '@/lib/kit';
import { Icon } from '@/components/ui/icon';
import { Reveal } from '@/components/ui/reveal';
import { Stagger, useScrollFade } from '@/components/ui/motion';

export type CompareOutputsVariant = 'blind-vote' | 'text-diff' | 'model-grid' | 'eval-table';

export interface CompareOutput {
  id: string;
  /** Model or version name, hidden until the vote in blind-vote */
  model: string;
  text: string;
  /** e.g. "1.8 s" */
  time?: string;
  /** e.g. "412 tokens" */
  tokens?: string;
  /** e.g. "0.4 credits" */
  cost?: string;
}

export interface EvalCase {
  id: string;
  prompt: string;
  /** Score per model id, 0 to 1 */
  scores: Record<string, number>;
  /** What the case checks, e.g. "Cites a source" */
  check?: string;
}

export interface CompareOutputsProps {
  variant?: CompareOutputsVariant;
  title?: string;
  /** The prompt both outputs answer */
  prompt?: string;
  outputs?: CompareOutput[];
  /** text-diff: the earlier and the later version */
  before?: { label: string; text: string };
  after?: { label: string; text: string };
  /** eval-table */
  models?: { id: string; name: string }[];
  cases?: EvalCase[];
  /** eval-table: score from which a case passes, 0 to 1 */
  passAt?: number;
  onVote?: (choice: string) => void;
  labels?: Partial<(typeof TEXTS)['en']>;
  className?: string;
}

const TEXTS = {
  en: {
    prompt: 'Prompt',
    answer: 'Answer {n}',
    which: 'Which answer is better?',
    better: '{n} is better',
    tie: 'Both are good',
    bad: 'Both are poor',
    revealed: 'Thanks. Here is which model wrote what.',
    again: 'Vote again',
    blindNote: 'Models stay hidden until you vote, so the name does not sway you.',
    youPicked: 'Your pick',
    changes: 'Changes',
    added: '{n} added',
    removed: '{n} removed',
    
    onlyChanges: 'Changes only',
    inline: 'Inline',
    sideBySide: 'Side by side',
    
    time: 'Time',
    tokens: 'Length',
    cost: 'Cost',
    pin: 'Mark as best',
    pinned: 'Best answer',
    copy: 'Copy',
    copied: 'Copied',
    case: 'Case',
    cases: '{n} cases',
    
    passed: '{n} of {total} passed',
    failing: 'Show failing only',
    pass: 'Pass',
    fail: 'Fail',
    scoreNote: 'Scores come from an automatic check. Read the failing answers before you decide.',
    model: 'Model',
    
    
  },
  de: {
    prompt: 'Auftrag',
    answer: 'Antwort {n}',
    which: 'Welche Antwort ist besser?',
    better: '{n} ist besser',
    tie: 'Beide gut',
    bad: 'Beide schwach',
    revealed: 'Danke. So verteilen sich die Modelle.',
    again: 'Neu abstimmen',
    blindNote: 'Die Modelle bleiben verborgen, bis du abstimmst, damit der Name nicht beeinflusst.',
    youPicked: 'Deine Wahl',
    changes: 'Änderungen',
    added: '{n} neu',
    removed: '{n} entfernt',
    
    onlyChanges: 'Nur Änderungen',
    inline: 'Im Text',
    sideBySide: 'Nebeneinander',
    
    time: 'Dauer',
    tokens: 'Länge',
    cost: 'Kosten',
    pin: 'Als beste markieren',
    pinned: 'Beste Antwort',
    copy: 'Kopieren',
    copied: 'Kopiert',
    case: 'Fall',
    cases: '{n} Fälle',
    
    passed: '{n} von {total} bestanden',
    failing: 'Nur nicht bestandene',
    pass: 'Bestanden',
    fail: 'Nicht bestanden',
    scoreNote: 'Die Werte kommen aus einer automatischen Prüfung. Lies die schwachen Antworten, bevor du entscheidest.',
    model: 'Modell',
    
    
  },
};

type Texts = (typeof TEXTS)['en'];
type P = CompareOutputsProps & { t: Texts };
const fill = (s: string, v: Record<string, string | number>) => s.replace(/\{(\w+)\}/g, (_, k) => String(v[k] ?? ''));

const CSS =
  '@keyframes cmp-up{from{opacity:0;transform:translateY(8px)}}@keyframes cmp-flip{0%{opacity:0;transform:rotateX(70deg)}100%{opacity:1;transform:none}}@keyframes cmp-bar{from{transform:scaleX(0)}}@keyframes cmp-hl{from{background-size:0 100%}}' +
  '@media (prefers-reduced-motion:no-preference){.cmp-up{animation:cmp-up .45s var(--ease-kit) var(--cmp-d,0ms) both}.cmp-flip{transform-origin:50% 0;animation:cmp-flip .55s var(--ease-kit) var(--cmp-d,0ms) both}.cmp-bar{transform-origin:left;animation:cmp-bar .9s var(--ease-kit) var(--cmp-d,0ms) both}.cmp-hl{animation:cmp-hl .7s var(--ease-kit) var(--cmp-d,0ms) both}}';

function Style() {
  return (
    <style href="kit-compare-outputs" precedence="default">
      {CSS}
    </style>
  );
}

/* ------------------------------------------------------------------ */
/* Shared parts                                                        */
/* ------------------------------------------------------------------ */

function Stage({ title, prompt, t, children, className, wide = true }: { title?: string; prompt?: string; t: Texts; children: React.ReactNode; className?: string; wide?: boolean }) {
  return (
    <section className={cn('kit-glow relative overflow-clip bg-surface py-12 sm:py-16 lg:py-20', className)}>
      <div className="kit-container">
        <div className={cn('mx-auto w-full', wide ? 'max-w-[68rem]' : 'max-w-[48rem]')}>
          {(title || prompt) && (
            <Reveal className="mb-6 sm:mb-8">
              {title && <h2 className="text-balance text-[22px] font-semibold leading-tight tracking-[-0.02em] text-foreground sm:text-[26px]">{title}</h2>}
              {prompt && (
                <p className="mt-3 flex max-w-[44rem] items-start gap-2.5 rounded-xl border border-border bg-card px-4 py-3 text-[14.5px] leading-relaxed text-foreground shadow-[var(--shadow-sm)]">
                  <span className="mt-0.5 shrink-0 text-[12px] font-medium text-muted-foreground">{t.prompt}</span>
                  {prompt}
                </p>
              )}
            </Reveal>
          )}
          {children}
        </div>
      </div>
    </section>
  );
}

function Paragraphs({ text, className }: { text: string; className?: string }) {
  return (
    <div className={cn('space-y-3 text-pretty text-[14.5px] leading-relaxed text-foreground', className)}>
      {text.split(/\n{2,}/).map((p, i) => (
        <p key={i} className="whitespace-pre-line">
          {p.split(/(\*\*[^*]+\*\*)/g).map((x, j) => (x.startsWith('**') ? <strong key={j} className="font-semibold">{x.slice(2, -2)}</strong> : <Fragment key={j}>{x}</Fragment>))}
        </p>
      ))}
    </div>
  );
}

const Letter = ({ i }: { i: number }) => String.fromCharCode(65 + i);

/* ------------------------------------------------------------------ */
/* blind-vote                                                          */
/* ------------------------------------------------------------------ */

function BlindVote({ t, title, prompt, outputs = [], onVote, className }: P) {
  const pair = outputs.slice(0, 2);
  const [vote, setVote] = useState<string | null>(null);
  const choices = [...pair.map((o, i) => ({ id: o.id, label: fill(t.better, { n: Letter({ i }) }) })), { id: 'tie', label: t.tie }, { id: 'bad', label: t.bad }];
  const pick = (id: string) => {
    setVote(id);
    onVote?.(id);
  };
  return (
    <Stage title={title} prompt={prompt} t={t} className={className}>
      <Stagger step={120} className="grid grid-cols-1 gap-3 md:grid-cols-2 md:gap-4">
        {pair.map((o, i) => {
          const won = vote === o.id;
          const lost = vote && vote !== o.id && vote !== 'tie' && vote !== 'bad';
          return (
            <article key={o.id} className={cn('flex h-full flex-col rounded-2xl border bg-card shadow-[var(--highlight),var(--shadow-sm)] transition-[border-color,opacity,box-shadow] duration-500 ease-[var(--ease-kit)]', won ? 'border-[color-mix(in_oklab,var(--primary)_60%,var(--border))] shadow-[var(--highlight),var(--shadow-md),0_0_0_3px_color-mix(in_oklab,var(--primary)_16%,transparent)]' : 'border-border', lost && 'opacity-70')}>
              <header className="flex h-12 items-center gap-2.5 border-b border-border px-4 [perspective:400px]">
                <span className={cn('grid size-6 place-items-center rounded-md text-[12px] font-semibold', won ? 'bg-primary text-primary-foreground' : 'bg-[color-mix(in_oklab,var(--foreground)_8%,transparent)] text-foreground')}>
                  <Letter i={i} />
                </span>
                {vote ? (
                  <span key="m" className="cmp-flip text-[13.5px] font-medium text-foreground" style={{ '--cmp-d': `${i * 120}ms` } as React.CSSProperties}>
                    {o.model}
                  </span>
                ) : (
                  <span className="text-[13.5px] font-medium text-muted-foreground">{fill(t.answer, { n: Letter({ i }) })}</span>
                )}
                {won && (
                  <span className="cmp-up ml-auto inline-flex items-center gap-1 rounded-md bg-[color-mix(in_oklab,var(--primary)_14%,var(--card))] px-2 py-1 text-[11.5px] font-medium text-[color-mix(in_oklab,var(--primary)_75%,var(--foreground))]">
                    <Icon name="check" size={12} strokeWidth={2.2} />
                    {t.youPicked}
                  </span>
                )}
              </header>
              <div className="flex-1 p-4 sm:p-5">
                <Paragraphs text={o.text} />
              </div>
              {(o.time || o.tokens) && (
                <footer className="flex gap-4 border-t border-border px-4 py-2.5 text-[12px] tabular-nums text-muted-foreground">
                  {o.time && <span>{o.time}</span>}
                  {o.tokens && <span>{o.tokens}</span>}
                </footer>
              )}
            </article>
          );
        })}
      </Stagger>
      <Reveal delay={200}>
        <div className="mt-4 rounded-2xl border border-border bg-card p-4 shadow-[var(--highlight),var(--shadow-sm)] sm:p-5" aria-live="polite">
          {vote ? (
            <div className="cmp-up flex flex-col gap-3 sm:flex-row sm:items-center sm:justify-between">
              <p className="flex items-center gap-2 text-[14px] text-foreground">
                <Icon name="check-circle" size={17} className="text-success-foreground" />
                {t.revealed}
              </p>
              <button type="button" onClick={() => setVote(null)} className="inline-flex h-11 items-center justify-center gap-1.5 rounded-lg border border-border bg-card px-4 text-[13.5px] font-medium text-foreground hover:bg-accent sm:h-9">
                <Icon name="refresh" size={15} />
                {t.again}
              </button>
            </div>
          ) : (
            <>
              <p className="text-[14px] font-medium text-foreground">{t.which}</p>
              <div className="mt-3 grid grid-cols-2 gap-2 sm:grid-cols-4">
                {choices.map((c, i) => (
                  <button key={c.id} type="button" onClick={() => pick(c.id)} className={cn('inline-flex min-h-11 items-center justify-center gap-1.5 rounded-lg border px-3 text-[13.5px] font-medium transition-colors', i < 2 ? 'border-[color-mix(in_oklab,var(--primary)_40%,var(--border))] text-foreground hover:bg-[color-mix(in_oklab,var(--primary)_10%,transparent)]' : 'border-border text-muted-foreground hover:bg-accent hover:text-foreground')}>
                    {i < 2 && <Icon name={i === 0 ? 'arrow-left' : 'arrow-right'} size={15} className="max-md:rotate-90" />}
                    {c.label}
                  </button>
                ))}
              </div>
              <p className="mt-3 flex items-start gap-1.5 text-[12.5px] text-muted-foreground">
                <Icon name="eye-off" size={14} className="mt-px shrink-0" />
                {t.blindNote}
              </p>
            </>
          )}
        </div>
      </Reveal>
    </Stage>
  );
}

/* ------------------------------------------------------------------ */
/* text-diff                                                           */
/* ------------------------------------------------------------------ */

type Op = { kind: 'same' | 'add' | 'del'; text: string };

/**
 * Word diff by longest common subsequence, enough for answers of a few hundred
 * words. Changed words between two unchanged ones form one region: first what
 * goes, then what comes, so a rewrite reads as one swap and not as confetti.
 */
function diffWords(a: string, b: string): Op[] {
  const A = a.trim().split(/\s+/).filter(Boolean);
  const B = b.trim().split(/\s+/).filter(Boolean);
  const n = A.length;
  const m = B.length;
  const L = Array.from({ length: n + 1 }, () => new Uint16Array(m + 1));
  for (let i = n - 1; i >= 0; i--) for (let j = m - 1; j >= 0; j--) L[i][j] = A[i] === B[j] ? L[i + 1][j + 1] + 1 : Math.max(L[i + 1][j], L[i][j + 1]);
  const out: Op[] = [];
  let dels: string[] = [];
  let adds: string[] = [];
  const flush = () => {
    if (dels.length) out.push({ kind: 'del', text: dels.join(' ') });
    if (adds.length) out.push({ kind: 'add', text: adds.join(' ') });
    dels = [];
    adds = [];
  };
  const same = (w: string) => {
    flush();
    const last = out[out.length - 1];
    if (last?.kind === 'same') last.text += ` ${w}`;
    else out.push({ kind: 'same', text: w });
  };
  let i = 0;
  let j = 0;
  while (i < n && j < m) {
    if (A[i] === B[j]) same(A[i++]), j++;
    else if (L[i + 1][j] >= L[i][j + 1]) dels.push(A[i++]);
    else adds.push(B[j++]);
  }
  while (i < n) dels.push(A[i++]);
  while (j < m) adds.push(B[j++]);
  flush();
  return out;
}

const count = (ops: Op[], k: Op['kind']) => ops.filter((o) => o.kind === k).reduce((s, o) => s + o.text.split(' ').length, 0);

/** Joins the parts with single spaces, the marks never swallow a space */
function Joined({ ops, render }: { ops: Op[]; render: (o: Op, i: number) => React.ReactNode }) {
  return (
    <>
      {ops.map((o, i) => (
        <Fragment key={i}>
          {i > 0 && ' '}
          {render(o, i)}
        </Fragment>
      ))}
    </>
  );
}

function TextDiff({ t, title, prompt, before, after, className }: P) {
  const ops = useMemo(() => diffWords(before?.text ?? '', after?.text ?? ''), [before?.text, after?.text]);
  const [mode, setMode] = useState<'inline' | 'split'>('inline');
  const [only, setOnly] = useState(false);
  const add = count(ops, 'add');
  const del = count(ops, 'del');
  const shown = only ? ops.map((o) => (o.kind === 'same' && o.text.split(' ').length > 8 ? { ...o, text: `${o.text.split(' ').slice(0, 3).join(' ')} … ${o.text.split(' ').slice(-3).join(' ')}` } : o)) : ops;
  const del_ = 'rounded-[3px] bg-[color-mix(in_oklab,var(--destructive)_14%,transparent)] text-muted-foreground line-through decoration-[color-mix(in_oklab,var(--destructive)_65%,transparent)] [box-decoration-break:clone]';
  const add_ = 'cmp-hl rounded-[3px] bg-[linear-gradient(color-mix(in_oklab,var(--success)_20%,transparent),color-mix(in_oklab,var(--success)_20%,transparent))] bg-no-repeat text-foreground no-underline [box-decoration-break:clone] [background-size:100%_100%]';
  const seg = (k: 'inline' | 'split', label: string) => (
    <button type="button" aria-pressed={mode === k} onClick={() => setMode(k)} className={cn('inline-flex h-11 items-center rounded-md px-3 text-[13px] font-medium transition-colors sm:h-8', mode === k ? 'bg-card text-foreground shadow-[var(--highlight),var(--shadow-sm)]' : 'text-muted-foreground hover:text-foreground')}>
      {label}
    </button>
  );
  return (
    <Stage title={title} prompt={prompt} t={t} className={className} wide={mode === 'split'}>
      <Reveal delay={60}>
        <div className="mb-3 flex flex-wrap items-center gap-x-4 gap-y-2">
          <p className="mr-auto flex items-center gap-3 text-[13px] tabular-nums">
            <span className="font-medium text-foreground">{t.changes}</span>
            <span className="text-success-foreground">+{fill(t.added, { n: add })}</span>
            <span className="text-destructive">−{fill(t.removed, { n: del })}</span>
          </p>
          <label className="inline-flex min-h-11 cursor-pointer items-center gap-2 text-[13px] text-muted-foreground sm:min-h-8">
            <input type="checkbox" checked={only} onChange={(e) => setOnly(e.target.checked)} className="size-4 accent-[var(--primary)]" />
            {t.onlyChanges}
          </label>
          <div role="group" aria-label={t.changes} className="flex gap-1 rounded-lg bg-[color-mix(in_oklab,var(--foreground)_5%,transparent)] p-1 max-md:hidden">
            {seg('inline', t.inline)}
            {seg('split', t.sideBySide)}
          </div>
        </div>
        {mode === 'inline' ? (
          <article className="rounded-2xl border border-border bg-card shadow-[var(--highlight),var(--shadow-sm)]">
            <header className="flex flex-wrap items-center gap-2 border-b border-border px-4 py-3 text-[12.5px] text-muted-foreground sm:px-5">
              <span className="rounded-md bg-[color-mix(in_oklab,var(--destructive)_12%,transparent)] px-1.5 py-0.5">{before?.label}</span>
              <Icon name="arrow-right" size={13} />
              <span className="rounded-md bg-[color-mix(in_oklab,var(--success)_14%,transparent)] px-1.5 py-0.5 text-foreground">{after?.label}</span>
            </header>
            <p className="p-4 text-pretty text-[15px] leading-[1.85] text-foreground sm:p-6">
              <Joined ops={shown} render={(o, i) => (o.kind === 'same' ? <span className={only ? 'text-muted-foreground' : undefined}>{o.text}</span> : o.kind === 'del' ? <del className={del_}>{o.text}</del> : <ins className={add_} style={{ '--cmp-d': `${200 + i * 40}ms` } as React.CSSProperties}>{o.text}</ins>)} />
            </p>
          </article>
        ) : (
          <div className="grid grid-cols-2 gap-4">
            {[
              { side: before, keep: 'del' as const },
              { side: after, keep: 'add' as const },
            ].map(({ side, keep }) => (
              <article key={keep} className="rounded-2xl border border-border bg-card shadow-[var(--highlight),var(--shadow-sm)]">
                <header className="border-b border-border px-5 py-3 text-[12.5px] font-medium text-muted-foreground">{side?.label}</header>
                <p className="p-5 text-pretty text-[14.5px] leading-[1.8] text-foreground">
                  <Joined ops={shown.filter((o) => o.kind === 'same' || o.kind === keep)} render={(o) => (o.kind === 'same' ? <span className={only ? 'text-muted-foreground' : undefined}>{o.text}</span> : keep === 'del' ? <del className={del_}>{o.text}</del> : <ins className={add_}>{o.text}</ins>)} />
                </p>
              </article>
            ))}
          </div>
        )}
      </Reveal>
    </Stage>
  );
}

/* ------------------------------------------------------------------ */
/* model-grid                                                          */
/* ------------------------------------------------------------------ */

function ModelGrid({ t, title, prompt, outputs = [], className }: P) {
  const [best, setBest] = useState<string | null>(null);
  const [copied, setCopied] = useState<string | null>(null);
  const [active, setActive] = useState(0);
  const [row, fade] = useScrollFade<HTMLDivElement>();
  const scroller = row as React.MutableRefObject<HTMLDivElement | null>;
  const go = (i: number) => {
    const el = scroller.current?.children[0]?.children[i] as HTMLElement | undefined;
    el?.scrollIntoView({ behavior: prefersReducedMotion() ? 'auto' : 'smooth', inline: 'start', block: 'nearest' });
  };
  return (
    <Stage title={title} prompt={prompt} t={t} className={className}>
      <div
        ref={row}
        tabIndex={0}
        aria-label={t.model}
        onScroll={(e) => {
          const el = e.currentTarget;
          const w = (el.children[0]?.children[0] as HTMLElement | undefined)?.offsetWidth ?? 1;
          setActive(Math.round(el.scrollLeft / (w + 12)));
        }}
        className={cn('kit-no-scrollbar -mx-6 snap-x snap-mandatory scroll-px-6 overflow-x-auto px-6 lg:mx-0 lg:overflow-visible lg:px-0', fade, 'lg:[mask-image:none]')}
      >
        <Stagger step={90} className={cn('flex w-max gap-3 lg:grid lg:w-auto', outputs.length >= 4 ? 'lg:grid-cols-4' : 'lg:grid-cols-3')}>
          {outputs.map((o) => {
            const on = best === o.id;
            return (
              <article key={o.id} className={cn('flex w-[min(20rem,calc(100vw-4.5rem))] snap-start flex-col rounded-2xl border bg-card shadow-[var(--highlight),var(--shadow-sm)] transition-[border-color,box-shadow] duration-300 lg:w-auto', on ? 'border-[color-mix(in_oklab,var(--primary)_60%,var(--border))] shadow-[var(--highlight),var(--shadow-md),0_0_0_3px_color-mix(in_oklab,var(--primary)_16%,transparent)]' : 'border-border')}>
                <header className="flex items-center justify-between gap-2 border-b border-border px-4 py-3">
                  <span className="truncate text-[14px] font-semibold text-foreground">{o.model}</span>
                  {on && (
                    <span className="cmp-up inline-flex shrink-0 items-center gap-1 rounded-md bg-[color-mix(in_oklab,var(--primary)_14%,var(--card))] px-1.5 py-0.5 text-[11px] font-medium text-[color-mix(in_oklab,var(--primary)_75%,var(--foreground))]">
                      <Icon name="star" size={11} />
                      {t.pinned}
                    </span>
                  )}
                </header>
                <div className="flex-1 p-4">
                  <Paragraphs text={o.text} className="text-[14px]" />
                </div>
                <dl className="grid grid-cols-3 gap-px border-y border-border bg-border text-[12px]">
                  {[
                    [t.time, o.time],
                    [t.tokens, o.tokens],
                    [t.cost, o.cost],
                  ].map(([k, v]) => (
                    <div key={k} className="bg-card px-3 py-2">
                      <dt className="text-subtle-foreground">{k}</dt>
                      <dd className="mt-0.5 truncate tabular-nums text-foreground">{v ?? ''}</dd>
                    </div>
                  ))}
                </dl>
                <footer className="flex gap-1.5 p-2.5">
                  <button type="button" aria-pressed={on} onClick={() => setBest(on ? null : o.id)} className={cn('inline-flex h-11 flex-1 items-center justify-center gap-1.5 rounded-lg text-[13px] font-medium transition-colors sm:h-9', on ? 'bg-[color-mix(in_oklab,var(--primary)_14%,var(--card))] text-[color-mix(in_oklab,var(--primary)_75%,var(--foreground))]' : 'border border-border text-foreground hover:bg-accent')}>
                    <Icon name="star" size={14} />
                    {on ? t.pinned : t.pin}
                  </button>
                  <button
                    type="button"
                    aria-label={copied === o.id ? t.copied : t.copy}
                    onClick={() => {
                      navigator.clipboard?.writeText(o.text).catch(() => {});
                      setCopied(o.id);
                      window.setTimeout(() => setCopied((c) => (c === o.id ? null : c)), 1600);
                    }}
                    className="grid size-11 shrink-0 place-items-center rounded-lg border border-border text-muted-foreground hover:bg-accent hover:text-foreground sm:size-9"
                  >
                    <Icon name={copied === o.id ? 'check' : 'copy'} size={15} className={copied === o.id ? 'cmp-up text-success-foreground' : undefined} />
                  </button>
                </footer>
              </article>
            );
          })}
        </Stagger>
      </div>
      <div className="mt-4 flex items-center justify-center gap-1 lg:hidden">
        {outputs.map((o, i) => (
          <button key={o.id} type="button" aria-label={o.model} aria-current={i === active} onClick={() => go(i)} className="grid size-11 place-items-center">
            <span className={cn('h-1.5 rounded-full transition-[width,background-color] duration-300', i === active ? 'w-5 bg-primary' : 'w-1.5 bg-[color-mix(in_oklab,var(--foreground)_25%,transparent)]')} />
          </button>
        ))}
      </div>
    </Stage>
  );
}

/* ------------------------------------------------------------------ */
/* eval-table                                                          */
/* ------------------------------------------------------------------ */

function EvalTable({ t, title, prompt, models = [], cases = [], passAt = 0.7, className }: P) {
  const locale = useLocale();
  const [failing, setFailing] = useState(false);
  const [phone, setPhone] = useState(models[0]?.id);
  const pct = (v: number) => (locale === 'de' ? `${Math.round(v * 100)} %` : `${Math.round(v * 100)}%`);
  const avg = (id: string) => (cases.length ? cases.reduce((s, c) => s + (c.scores[id] ?? 0), 0) / cases.length : 0);
  const passed = (id: string) => cases.filter((c) => (c.scores[id] ?? 0) >= passAt).length;
  const shown = failing ? cases.filter((c) => models.some((m) => (c.scores[m.id] ?? 0) < passAt)) : cases;
  // Best by the shown percentage, a tie goes to more passed cases
  const rank = (id: string) => Math.round(avg(id) * 100) * 100 + passed(id);
  const topId = models.reduce<string | undefined>((b, m) => (!b || rank(m.id) > rank(b) ? m.id : b), undefined);
  const cell = (v: number) => (
    <span className={cn('inline-flex min-w-[3.25rem] items-center justify-center gap-1 rounded-md px-1.5 py-1 text-[12.5px] font-medium tabular-nums', v >= passAt ? 'bg-[color-mix(in_oklab,var(--success)_14%,transparent)] text-success-foreground' : 'bg-[color-mix(in_oklab,var(--destructive)_12%,transparent)] text-destructive')}>
      <Icon name={v >= passAt ? 'check' : 'x'} size={12} strokeWidth={2.2} />
      <span className="sr-only">{v >= passAt ? t.pass : t.fail}</span>
      {pct(v)}
    </span>
  );
  const tabs = useRef<(HTMLButtonElement | null)[]>([]);

  return (
    <Stage title={title} prompt={prompt} t={t} className={className}>
      <Stagger step={90} className={cn('mb-4 grid gap-2 sm:gap-3', models.length >= 3 ? 'grid-cols-3' : 'grid-cols-2')}>
        {models.map((m, i) => (
          <div key={m.id} className={cn('min-w-0 rounded-2xl border bg-card p-3 shadow-[var(--highlight),var(--shadow-sm)] sm:p-4', m.id === topId ? 'border-[color-mix(in_oklab,var(--primary)_50%,var(--border))]' : 'border-border')}>
            <p className="flex items-center justify-between gap-2 text-[13px] font-medium text-foreground">
              {m.name}
              {m.id === topId && <Icon name="trophy" size={15} className="text-primary" />}
            </p>
            <p className="mt-2 text-[20px] font-semibold leading-none tracking-[-0.02em] tabular-nums text-foreground sm:text-[26px]">{pct(avg(m.id))}</p>
            <div className="mt-3 h-1.5 overflow-hidden rounded-full bg-[color-mix(in_oklab,var(--foreground)_9%,transparent)]">
              <span className="cmp-bar block h-full rounded-full bg-primary" style={{ width: `${round(avg(m.id) * 100, 1)}%`, '--cmp-d': `${200 + i * 120}ms` } as React.CSSProperties} />
            </div>
            <p className="mt-2 text-[11.5px] leading-snug tabular-nums text-muted-foreground sm:text-[12px]">{fill(t.passed, { n: passed(m.id), total: cases.length })}</p>
          </div>
        ))}
      </Stagger>
      <Reveal delay={160}>
        <div className="rounded-2xl border border-border bg-card shadow-[var(--highlight),var(--shadow-sm)]">
          <div className="flex flex-wrap items-center justify-between gap-3 border-b border-border px-4 py-2.5 sm:px-5">
            <p className="text-[13px] font-medium text-foreground">
              {fill(t.cases, { n: cases.length })}
            </p>
            <label className="inline-flex min-h-11 cursor-pointer items-center gap-2 text-[13px] text-muted-foreground sm:min-h-8">
              <input type="checkbox" checked={failing} onChange={(e) => setFailing(e.target.checked)} className="size-4 accent-[var(--primary)]" />
              {t.failing}
            </label>
          </div>
          {/* Phones: one model at a time */}
          <div className="md:hidden">
            <div role="tablist" aria-label={t.model} className="flex gap-1 border-b border-border p-2">
              {models.map((m, i) => (
                <button
                  key={m.id}
                  ref={(el) => {
                    tabs.current[i] = el;
                  }}
                  type="button"
                  role="tab"
                  aria-selected={phone === m.id}
                  tabIndex={phone === m.id ? 0 : -1}
                  onClick={() => setPhone(m.id)}
                  onKeyDown={(e) => {
                    const d = e.key === 'ArrowRight' ? 1 : e.key === 'ArrowLeft' ? -1 : 0;
                    if (!d) return;
                    e.preventDefault();
                    const n = (i + d + models.length) % models.length;
                    setPhone(models[n].id);
                    tabs.current[n]?.focus();
                  }}
                  className={cn('min-h-11 flex-1 truncate rounded-lg px-2 text-[12.5px] font-medium', phone === m.id ? 'bg-accent text-foreground' : 'text-muted-foreground')}
                >
                  {m.name}
                </button>
              ))}
            </div>
            <ul role="tabpanel" className="divide-y divide-border">
              {shown.map((c) => (
                <li key={c.id} className="flex items-start justify-between gap-3 px-4 py-3">
                  <span className="min-w-0">
                    <span className="block text-[13.5px] leading-snug text-foreground">{c.prompt}</span>
                    {c.check && <span className="mt-0.5 block text-[12px] text-muted-foreground">{c.check}</span>}
                  </span>
                  {cell(c.scores[phone ?? ''] ?? 0)}
                </li>
              ))}
            </ul>
          </div>
          <table className="hidden w-full border-collapse text-left md:table">
            <caption className="sr-only">{title}</caption>
            <thead>
              <tr className="border-b border-border text-[12px] text-muted-foreground">
                <th scope="col" className="px-5 py-2.5 font-medium">
                  {t.case}
                </th>
                {models.map((m) => (
                  <th key={m.id} scope="col" className="w-32 px-3 py-2.5 text-center font-medium">
                    {m.name}
                  </th>
                ))}
              </tr>
            </thead>
            <tbody>
              {shown.map((c, ri) => (
                <tr key={c.id} className="cmp-up border-b border-border last:border-0" style={{ '--cmp-d': `${ri * 40}ms` } as React.CSSProperties}>
                  <th scope="row" className="px-5 py-3 font-normal">
                    <span className="block text-[13.5px] leading-snug text-foreground">{c.prompt}</span>
                    {c.check && <span className="mt-0.5 block text-[12px] text-muted-foreground">{c.check}</span>}
                  </th>
                  {models.map((m) => (
                    <td key={m.id} className="px-3 py-3 text-center">
                      {cell(c.scores[m.id] ?? 0)}
                    </td>
                  ))}
                </tr>
              ))}
            </tbody>
          </table>
        </div>
      </Reveal>
      <p className="mt-3 flex items-start gap-2 text-[12.5px] text-muted-foreground">
        <Icon name="info" size={14} className="mt-px shrink-0" />
        {t.scoreNote}
      </p>
    </Stage>
  );
}

/* ------------------------------------------------------------------ */

/**
 * Comparing AI outputs: a blind vote between two answers that reveals the
 * models afterwards, a word diff between two versions, answers of several
 * models side by side with time, length and cost, and an evaluation table.
 */
export function CompareOutputs({ variant = 'blind-vote', labels, ...props }: CompareOutputsProps) {
  const t = useTexts(TEXTS, labels);
  const p: P = { ...props, t };
  const body = (() => {
    switch (variant) {
      case 'text-diff':
        return <TextDiff {...p} />;
      case 'model-grid':
        return <ModelGrid {...p} />;
      case 'eval-table':
        return <EvalTable {...p} />;
      default:
        return <BlindVote {...p} />;
    }
  })();
  return (
    <>
      <Style />
      {body}
    </>
  );
}

Frequent questions

How do I install Compare outputs?

With aiw add @ai-website/compare-outputs. The command puts components/ui/compare-outputs.tsx into your project, the code is yours from then on. It brings along icon, reveal and motion. It needs no further npm packages.

Which variants does Compare outputs have?

4 variants, Blind vote, Text diff, Models side by side and Evaluation table. You pick one with the variant prop, for example variant="text-diff".

Can I translate the texts of Compare outputs?

Yes. Compare outputs carries 31 texts of its own in English and German, such as “Prompt”, “Which answer is better?” and “Both are good”. The language follows the KitProvider, single texts change through the labels prop.

Can an AI assistant build with Compare outputs?

Yes. Through the MCP server of the aiw CLI, Claude, Cursor, GitHub Copilot, Windsurf, Codex and Gemini find Compare outputs with aiw_components and install it with aiw_add @ai-website/compare-outputs. Name the variant in your prompt, for example variant="blind-vote", and the assistant writes your content straight into the props.

Can I adjust Compare outputs without code?

Yes, in the Studio. All 4 variants are editable there, texts, lists and switches. The Studio writes the matching CompareOutputs code to copy.

Often searched

Jump to

Copied to the clipboard