Skip to content
UI Kit
Accent color

Multimodal AI input

Input for an AI that reads more than text. A drop zone with limits per kind, a toolbar for image, file, camera, screenshot and voice, a voice note with level, timer and transcript, and a camera or screen capture where you mark the part that matters. Text features like commands and mentions live in prompt-input.

4 variantsOne of 5 components in AI and chat

Show the assistant, not only tell it

Many questions are easier to show than to type, a screenshot of an error, a photo of a receipt, a voice note on the way. Multimodal input lets an AI assistant take all of that and says up front what it will read, so nobody uploads a file in vain.

Drop zone is a large target for files from tablets on, with what each kind may weigh and what gets read, chips show progress, size and why a file is refused, and phones get two buttons for photo and files. Media toolbar puts one button per kind of media in the field, a screenshot lands as a chip with a flash and the voice button records right in the bar. Voice note records with live level, timer and time left, then sends as audio or inserts as text, the transcript appears while you speak and hard to hear words are marked. Camera and screen opens a viewfinder with guides or a window to capture, then you drag over the part that matters and attach it.

State the limits before the upload, not in an error after it. Let people crop a screenshot to the part that matters, the answer gets better and the request smaller. Text features like slash commands and mentions live in Prompt input, the two work side by side.

Variants

Drop zoneNew in 1.2

A large target for files from tablets on, with what each kind may weigh and what gets read. Chips show progress, size and why a file is refused. On phones two buttons for photo and files.

Media toolbarNew in 1.2

One button per kind of media in the field. A screenshot lands as a chip with a flash, the voice button records right in the bar. Below, what the assistant reads per kind and how much room is left.

Voice noteNew in 1.2

A recorder with live level, timer and the time left. Pause, discard or keep, then send as audio or insert as text. The transcript appears as you speak, hard to hear words are marked.

Camera and screenNew in 1.2

A viewfinder with guides and a shutter, or a window to capture. After the picture, drag over the part that matters, then attach it to the message next to it.

Details

What Multimodal input brings along, read from the registry and the source.

Category
AI and chat
Area
App
Variants
4
Version
1.0.0 · since 1.2.0
File
components/ui/multimodal-input.tsx
Brings along
utilskiticonreveal
npm packages
None
Languages
English and German, 67 texts of its own
Studio
Adjust in code
In collections
AI product

Props

The MultimodalInputProps interface from components/ui/multimodal-input.tsx, with the comments from the code.

  • variantoptional
    MultimodalInputVariant

    dropzonemedia-toolbarvoice-notecamera-capture

  • titleoptional
    string
  • descriptionoptional
    string
  • placeholderoptional
    string
  • defaultValueoptional
    string

    Text already in the field

  • attachmentsoptional
    MediaAttachment[]
  • maxFilesoptional
    number

    Most attachments per message

  • maxSizeMboptional
    number

    Largest file in MB, unless a kind sets its own

  • supportoptional
    MediaSupport[]

    What the model reads per kind

  • maxSecondsoptional
    number

    voice-note: longest recording in seconds

  • transcriptoptional
    string

    voice-note: the text heard, shown as the transcript

  • autoStartoptional
    boolean

    voice-note: start recording once in view

  • previewoptional
    React.ReactNode

    camera-capture: the picture the camera sees

  • screenoptional
    React.ReactNode

    camera-capture: the picture a screenshot shows

  • windowsoptional
    CaptureWindow[]

    camera-capture: windows to choose in screenshot mode

  • noteoptional
    string

    Line under the field, e.g. what happens with files and audio

  • onSubmitoptional
    (value: MultimodalSubmit) => void

Built-in texts

Multimodal input writes these texts itself. The language follows the KitProvider, the labels prop replaces single ones.

Show all 67 texts
messageEnglish MessageGerman Nachricht
placeholderEnglish Ask about your files, photos or a recordingGerman Frag zu deinen Dateien, Fotos oder einer Aufnahme
sendEnglish SendGerman Senden
sentEnglish Sent with {n} attachmentsGerman Gesendet mit {n} Anhängen
sent1English Sent with 1 attachmentGerman Gesendet mit 1 Anhang
sentTextEnglish SentGerman Gesendet
attachmentsEnglish AttachmentsGerman Anhänge
removeEnglish Remove {name}German {name} entfernen
uploadingEnglish Uploading, {n}%German Wird hochgeladen, {n} %
readyEnglish ReadyGerman Bereit
tooLargeEnglish Larger than {mb} MBGerman Größer als {mb} MB
tooLongEnglish Longer than {v}German Länger als {v}
tooManyEnglish Up to {n} attachments per message. The rest was left out.German Bis zu {n} Anhänge je Nachricht. Der Rest wurde weggelassen.
unsupportedEnglish This file type cannot be readGerman Diese Dateiart kann nicht gelesen werden
dropEnglish Drop to attachGerman Zum Anhängen ablegen
dropHintEnglish Drop files here or choose themGerman Dateien hier ablegen oder auswählen
chooseEnglish Choose filesGerman Dateien wählen
takePhotoEnglish Take a photoGerman Foto aufnehmen
limitsEnglish {count} of {max} attachments · up to {mb} MB eachGerman {count} von {max} Anhängen · je bis {mb} MB
totalEnglish {used} of {max} MBGerman {used} von {max} MB
imageEnglish ImageGerman Bild
screenshotEnglish ScreenshotGerman Bildschirmfoto
pdfEnglish PDFGerman PDF
docEnglish DocumentGerman Dokument
sheetEnglish SpreadsheetGerman Tabelle
audioEnglish AudioGerman Audio
videoEnglish VideoGerman Video
fileEnglish FileGerman Datei
cameraEnglish CameraGerman Kamera
screenEnglish ScreenGerman Bildschirm
voiceEnglish VoiceGerman Sprache
addImageEnglish Add an imageGerman Bild hinzufügen
addFileEnglish Add a fileGerman Datei hinzufügen
useCameraEnglish Use the cameraGerman Kamera nutzen
captureScreenEnglish Take a screenshotGerman Bildschirmfoto aufnehmen
recordVoiceEnglish Record a voice noteGerman Sprachnachricht aufnehmen
readsEnglish What the assistant readsGerman Was der Assistent liest
upToEnglish up to {v}German bis {v}
recordEnglish Start recordingGerman Aufnahme starten
recordingEnglish RecordingGerman Nimmt auf
pausedEnglish PausedGerman Pausiert
pauseEnglish PauseGerman Pausieren
resumeEnglish ResumeGerman Weiter
stopEnglish Stop and keepGerman Beenden und behalten
discardEnglish DiscardGerman Verwerfen
levelEnglish Input levelGerman Eingangspegel
leftEnglish {v} leftGerman noch {v}
voiceNoteEnglish Voice noteGerman Sprachnachricht
asAudioEnglish Send as audioGerman Als Audio senden
asTextEnglish Insert as textGerman Als Text einfügen
transcriptEnglish TranscriptGerman Mitschrift
uncertainEnglish Words in dotted lines were hard to hear.German Gepunktet unterstrichene Wörter waren schwer zu verstehen.
micNoteEnglish The microphone runs only while the red light is on.German Das Mikrofon läuft nur, solange der rote Punkt leuchtet.
photoEnglish PhotoGerman Foto
shutterEnglish Take pictureGerman Foto machen
flipEnglish Switch cameraGerman Kamera wechseln
retakeEnglish Take againGerman Neu aufnehmen
attachEnglish AttachGerman Anhängen
attachedEnglish AttachedGerman Angehängt
markAreaEnglish Drag over the part the assistant should look atGerman Ziehe über den Teil, den sich der Assistent ansehen soll
wholePictureEnglish Whole pictureGerman Ganzes Bild
areaEnglish Marked areaGerman Markierter Bereich
pickWindowEnglish Choose what to captureGerman Wähl, was aufgenommen wird
captureEnglish CaptureGerman Aufnehmen
cameraNoteEnglish Pictures stay on this device until you send them.German Bilder bleiben auf diesem Gerät, bis du sie sendest.
viewfinderEnglish Camera viewGerman Kamerabild
howToEnglish Take a picture or capture a window|Drag over the part that matters|Attach it and ask your questionGerman Foto machen oder ein Fenster aufnehmen|Über den wichtigen Teil ziehen|Anhängen und deine Frage stellen

Accessibility

Read from the source of the component.

  • Multimodal input handles the keys Enter, Arrow up, Arrow down, Arrow left, Arrow right and Space itself.
  • Sets aria-checked, aria-label, aria-live, aria-valuemax, aria-valuemin and aria-valuenow, so screen readers announce purpose and state.
  • Uses the roles alert, group, meter, radio, radiogroup, shot and status.
  • Announces changes through a live region without moving focus.
  • Every field has a label, visible or for screen readers.
  • Adds text for screen readers where icons and graphics stand alone.
  • Shows a visible focus ring when used with the keyboard.
  • The server renders the starting state in full, JavaScript only adds the interaction.

Motion

  • Rises in briefly the first time it comes into view (kit-rise, useInView).
  • Moves with the scroll, its progress follows the position on the page.
  • Brings animations of its own, calmly timed and without wobble.
  • With reduced motion (prefers-reduced-motion) it stays still. 4 places in the code switch motion off then.

Source

The file aiw add puts into your project. From then on it is yours.

Show the file, 1,160 lines
components/ui/multimodal-input.tsx
'use client';

import { useEffect, useId, useRef, useState } from 'react';
import { cn, prefersReducedMotion, round, useInView, useLocale, useTexts } from '@/lib/kit';
import { Icon, type IconName } from '@/components/ui/icon';
import { Reveal } from '@/components/ui/reveal';

export type MultimodalInputVariant = 'dropzone' | 'media-toolbar' | 'voice-note' | 'camera-capture';

export type MediaKind = 'image' | 'screenshot' | 'pdf' | 'doc' | 'sheet' | 'audio' | 'video';

export interface MediaAttachment {
  id: string;
  name: string;
  kind: MediaKind;
  /** Size in bytes */
  size: number;
  /** 0 to 100, below 100 the file is still uploading */
  progress?: number;
  /** A message when the file cannot be used */
  error?: string;
  /** Audio and video length in seconds */
  duration?: number;
  /** Preview for images and screenshots, a drawn motif or your own <img> */
  media?: React.ReactNode;
}

/** What the model reads per kind of media, shown as plain words */
export interface MediaSupport {
  kind: MediaKind;
  /** e.g. "Reads text and tables" */
  reads: string;
  /** Largest file in MB */
  maxMb?: number;
  /** Longest audio or video in seconds */
  maxSeconds?: number;
}

/** camera-capture: a window to share in screenshot mode */
export interface CaptureWindow {
  id: string;
  label: string;
  icon?: IconName;
}

export interface MultimodalSubmit {
  text: string;
  attachments: MediaAttachment[];
}

export interface MultimodalInputProps {
  variant?: MultimodalInputVariant;
  title?: string;
  description?: string;
  placeholder?: string;
  /** Text already in the field */
  defaultValue?: string;
  attachments?: MediaAttachment[];
  /** Most attachments per message */
  maxFiles?: number;
  /** Largest file in MB, unless a kind sets its own */
  maxSizeMb?: number;
  /** What the model reads per kind */
  support?: MediaSupport[];
  /** voice-note: longest recording in seconds */
  maxSeconds?: number;
  /** voice-note: the text heard, shown as the transcript */
  transcript?: string;
  /** voice-note: start recording once in view */
  autoStart?: boolean;
  /** camera-capture: the picture the camera sees */
  preview?: React.ReactNode;
  /** camera-capture: the picture a screenshot shows */
  screen?: React.ReactNode;
  /** camera-capture: windows to choose in screenshot mode */
  windows?: CaptureWindow[];
  /** Line under the field, e.g. what happens with files and audio */
  note?: string;
  onSubmit?: (value: MultimodalSubmit) => void;
  labels?: Partial<(typeof TEXTS)['en']>;
  className?: string;
}

const TEXTS = {
  en: {
    message: 'Message',
    placeholder: 'Ask about your files, photos or a recording',
    send: 'Send',
    sent: 'Sent with {n} attachments',
    sent1: 'Sent with 1 attachment',
    sentText: 'Sent',
    attachments: 'Attachments',
    remove: 'Remove {name}',
    uploading: 'Uploading, {n}%',
    ready: 'Ready',
    tooLarge: 'Larger than {mb} MB',
    tooLong: 'Longer than {v}',
    tooMany: 'Up to {n} attachments per message. The rest was left out.',
    unsupported: 'This file type cannot be read',
    drop: 'Drop to attach',
    dropHint: 'Drop files here or choose them',
    choose: 'Choose files',
    takePhoto: 'Take a photo',
    limits: '{count} of {max} attachments · up to {mb} MB each',
    total: '{used} of {max} MB',
    image: 'Image',
    screenshot: 'Screenshot',
    pdf: 'PDF',
    doc: 'Document',
    sheet: 'Spreadsheet',
    audio: 'Audio',
    video: 'Video',
    file: 'File',
    camera: 'Camera',
    screen: 'Screen',
    voice: 'Voice',
    addImage: 'Add an image',
    addFile: 'Add a file',
    useCamera: 'Use the camera',
    captureScreen: 'Take a screenshot',
    recordVoice: 'Record a voice note',
    reads: 'What the assistant reads',
    upTo: 'up to {v}',
    record: 'Start recording',
    recording: 'Recording',
    paused: 'Paused',
    pause: 'Pause',
    resume: 'Resume',
    stop: 'Stop and keep',
    discard: 'Discard',
    level: 'Input level',
    left: '{v} left',
    voiceNote: 'Voice note',
    asAudio: 'Send as audio',
    asText: 'Insert as text',
    transcript: 'Transcript',
    uncertain: 'Words in dotted lines were hard to hear.',
    micNote: 'The microphone runs only while the red light is on.',
    photo: 'Photo',
    shutter: 'Take picture',
    flip: 'Switch camera',
    retake: 'Take again',
    attach: 'Attach',
    attached: 'Attached',
    markArea: 'Drag over the part the assistant should look at',
    wholePicture: 'Whole picture',
    area: 'Marked area',
    pickWindow: 'Choose what to capture',
    capture: 'Capture',
    cameraNote: 'Pictures stay on this device until you send them.',
    viewfinder: 'Camera view',
    howTo: 'Take a picture or capture a window|Drag over the part that matters|Attach it and ask your question',
  },
  de: {
    message: 'Nachricht',
    placeholder: 'Frag zu deinen Dateien, Fotos oder einer Aufnahme',
    send: 'Senden',
    sent: 'Gesendet mit {n} Anhängen',
    sent1: 'Gesendet mit 1 Anhang',
    sentText: 'Gesendet',
    attachments: 'Anhänge',
    remove: '{name} entfernen',
    uploading: 'Wird hochgeladen, {n} %',
    ready: 'Bereit',
    tooLarge: 'Größer als {mb} MB',
    tooLong: 'Länger als {v}',
    tooMany: 'Bis zu {n} Anhänge je Nachricht. Der Rest wurde weggelassen.',
    unsupported: 'Diese Dateiart kann nicht gelesen werden',
    drop: 'Zum Anhängen ablegen',
    dropHint: 'Dateien hier ablegen oder auswählen',
    choose: 'Dateien wählen',
    takePhoto: 'Foto aufnehmen',
    limits: '{count} von {max} Anhängen · je bis {mb} MB',
    total: '{used} von {max} MB',
    image: 'Bild',
    screenshot: 'Bildschirmfoto',
    pdf: 'PDF',
    doc: 'Dokument',
    sheet: 'Tabelle',
    audio: 'Audio',
    video: 'Video',
    file: 'Datei',
    camera: 'Kamera',
    screen: 'Bildschirm',
    voice: 'Sprache',
    addImage: 'Bild hinzufügen',
    addFile: 'Datei hinzufügen',
    useCamera: 'Kamera nutzen',
    captureScreen: 'Bildschirmfoto aufnehmen',
    recordVoice: 'Sprachnachricht aufnehmen',
    reads: 'Was der Assistent liest',
    upTo: 'bis {v}',
    record: 'Aufnahme starten',
    recording: 'Nimmt auf',
    paused: 'Pausiert',
    pause: 'Pausieren',
    resume: 'Weiter',
    stop: 'Beenden und behalten',
    discard: 'Verwerfen',
    level: 'Eingangspegel',
    left: 'noch {v}',
    voiceNote: 'Sprachnachricht',
    asAudio: 'Als Audio senden',
    asText: 'Als Text einfügen',
    transcript: 'Mitschrift',
    uncertain: 'Gepunktet unterstrichene Wörter waren schwer zu verstehen.',
    micNote: 'Das Mikrofon läuft nur, solange der rote Punkt leuchtet.',
    photo: 'Foto',
    shutter: 'Foto machen',
    flip: 'Kamera wechseln',
    retake: 'Neu aufnehmen',
    attach: 'Anhängen',
    attached: 'Angehängt',
    markArea: 'Ziehe über den Teil, den sich der Assistent ansehen soll',
    wholePicture: 'Ganzes Bild',
    area: 'Markierter Bereich',
    pickWindow: 'Wähl, was aufgenommen wird',
    capture: 'Aufnehmen',
    cameraNote: 'Bilder bleiben auf diesem Gerät, bis du sie sendest.',
    viewfinder: 'Kamerabild',
    howTo: 'Foto machen oder ein Fenster aufnehmen|Über den wichtigen Teil ziehen|Anhängen und deine Frage stellen',
  },
};

type Texts = (typeof TEXTS)['en'];
type P = MultimodalInputProps & { t: Texts };
const fill = (s: string, v: Record<string, string | number>) => s.replace(/\{(\w+)\}/g, (_, k) => String(v[k] ?? ''));

/** State seeded from a prop that starts over when the prop changes (the Studio edits props live) */
function useSynced<T>(value: T) {
  const [state, setState] = useState(value);
  const key = JSON.stringify(value, (k, v) => (k === 'media' ? undefined : v));
  const seen = useRef(key);
  if (seen.current !== key) {
    seen.current = key;
    setState(value);
  }
  return [state, setState] as const;
}

const CSS =
  '@keyframes mmi-up{from{opacity:0;transform:translateY(6px) scale(.97)}}@keyframes mmi-blink{50%{opacity:.25}}@keyframes mmi-flash{from{opacity:.85}to{opacity:0}}' +
  '@media (prefers-reduced-motion:no-preference){.mmi-up{animation:mmi-up .35s var(--ease-kit) var(--mmi-d,0ms) both}.mmi-blink{animation:mmi-blink 1.2s ease-in-out infinite}.mmi-flash{animation:mmi-flash .5s ease-out both}}';

function Style() {
  return (
    <style href="kit-multimodal-input" precedence="default">
      {CSS}
    </style>
  );
}

/* ------------------------------------------------------------------ */
/* Shared parts                                                        */
/* ------------------------------------------------------------------ */

const KIND: Record<MediaKind, { icon: IconName; tone: string }> = {
  image: { icon: 'image', tone: '--info' },
  screenshot: { icon: 'monitor', tone: '--info' },
  pdf: { icon: 'file', tone: '--destructive' },
  doc: { icon: 'file', tone: '--primary' },
  sheet: { icon: 'table', tone: '--success' },
  audio: { icon: 'mic', tone: '--warning' },
  video: { icon: 'film', tone: '--primary' },
};

const card = 'rounded-2xl border border-border bg-card shadow-[var(--highlight),var(--shadow-md)]';
const btn = 'inline-flex h-11 shrink-0 items-center justify-center gap-1.5 whitespace-nowrap rounded-lg px-3.5 text-[13px] font-medium transition-colors disabled:pointer-events-none disabled:opacity-50 sm:h-9';
const primaryBtn = cn(btn, 'bg-primary text-primary-foreground shadow-[inset_0_1px_0_rgb(255_255_255/0.18),var(--shadow-sm)] hover:bg-[color-mix(in_oklab,var(--primary)_88%,var(--background))]');
const quietBtn = cn(btn, 'border border-border bg-card text-foreground hover:bg-accent');
const toolBtn = 'relative grid size-11 shrink-0 place-items-center rounded-lg text-muted-foreground transition-colors hover:bg-accent hover:text-foreground disabled:opacity-40 sm:size-9';

function useFormat() {
  const locale = useLocale();
  const intl = locale === 'de' ? 'de-DE' : 'en-US';
  const nf1 = new Intl.NumberFormat(intl, { maximumFractionDigits: 1 });
  const size = (b: number) => (b >= 1e6 ? `${nf1.format(b / 1e6)} MB` : `${Math.max(1, Math.round(b / 1e3))} KB`);
  const mb = (b: number) => nf1.format(b / 1e6);
  const time = (s: number) => `${Math.floor(s / 60)}:${String(Math.floor(s % 60)).padStart(2, '0')}`;
  const pct = (n: number) => (locale === 'de' ? `${n} %` : `${n}%`);
  return { size, mb, time, pct, de: locale === 'de' };
}

function kindOf(file: File): MediaKind | null {
  const t = file.type;
  if (t.startsWith('image/')) return 'image';
  if (t.startsWith('audio/')) return 'audio';
  if (t.startsWith('video/')) return 'video';
  if (t === 'application/pdf') return 'pdf';
  if (/sheet|excel|csv/.test(t) || /\.(xlsx?|csv|numbers)$/i.test(file.name)) return 'sheet';
  if (/word|text|rtf|opendocument/.test(t) || /\.(docx?|txt|md|rtf|odt)$/i.test(file.name)) return 'doc';
  return null;
}

/** A ring that fills while a file uploads */
function Ring({ value, size = 20 }: { value: number; size?: number }) {
  const r = size / 2 - 2;
  const c = 2 * Math.PI * r;
  return (
    <svg aria-hidden width={size} height={size} viewBox={`0 0 ${size} ${size}`} className="-rotate-90">
      <circle cx={size / 2} cy={size / 2} r={r} fill="none" stroke="var(--track, color-mix(in oklab, var(--foreground) 12%, transparent))" strokeWidth="2.5" />
      <circle cx={size / 2} cy={size / 2} r={r} fill="none" stroke="var(--primary)" strokeWidth="2.5" strokeLinecap="round" strokeDasharray={round(c)} strokeDashoffset={round(c * (1 - value / 100))} className="transition-[stroke-dashoffset] duration-200" />
    </svg>
  );
}

/** The tile of a file: its picture or an icon in the tone of its kind */
function Thumb({ a, className }: { a: MediaAttachment; className?: string }) {
  const k = KIND[a.kind];
  return (
    <span className={cn('relative grid size-10 shrink-0 place-items-center overflow-hidden rounded-lg', className)} style={a.media ? undefined : { background: `color-mix(in oklab, var(${k.tone}) 15%, var(--card))`, color: `color-mix(in oklab, var(${k.tone}) 75%, var(--foreground))` }}>
      {a.media ? (
        <span data-clip-ok className="absolute inset-0 [&>*]:size-full [&>img]:object-cover">
          {a.media}
        </span>
      ) : (
        <Icon name={k.icon} size={17} />
      )}
    </span>
  );
}

/** One attachment as a chip with progress, error and remove */
function Chip({ a, t, onRemove, delay = 0 }: { a: MediaAttachment; t: Texts; onRemove: () => void; delay?: number }) {
  const f = useFormat();
  const loading = a.progress !== undefined && a.progress < 100 && !a.error;
  const meta = a.error ? a.error : loading ? fill(t.uploading, { n: Math.round(a.progress ?? 0) }) : [t[a.kind], a.duration ? f.time(a.duration) : null, f.size(a.size)].filter(Boolean).join(' · ');
  return (
    <li className="mmi-up flex w-[14rem] max-w-full shrink-0 sm:w-[15rem] items-center gap-2.5 rounded-xl border bg-card py-1.5 pl-1.5 pr-1 shadow-[var(--shadow-sm)]" style={{ '--mmi-d': `${delay}ms`, borderColor: a.error ? 'color-mix(in oklab, var(--destructive) 45%, var(--border))' : 'var(--border)' } as React.CSSProperties}>
      <Thumb a={a} className={loading ? 'opacity-60' : undefined} />
      <span className="min-w-0 flex-1">
        <span className="block truncate text-[12.5px] font-medium text-foreground">{a.name}</span>
        <span className={cn('block truncate text-[11.5px] tabular-nums', a.error ? 'text-destructive' : 'text-muted-foreground')} aria-live={loading ? 'off' : 'polite'}>
          {meta}
        </span>
      </span>
      {loading && <Ring value={a.progress ?? 0} />}
      <button type="button" aria-label={fill(t.remove, { name: a.name })} onClick={onRemove} className="relative grid size-8 shrink-0 place-items-center rounded-md text-muted-foreground transition-colors after:absolute after:-inset-1.5 hover:bg-accent hover:text-foreground">
        <Icon name="x" size={14} />
      </button>
    </li>
  );
}

/** Attachments with uploads that move on while in view */
function useAttachments(initial: MediaAttachment[], maxFiles: number, maxSizeMb: number, support: MediaSupport[], t: Texts) {
  const [items, setItems] = useSynced(initial);
  const [notice, setNotice] = useState('');
  const [ref, inView] = useInView<HTMLDivElement>();
  const seq = useRef(0);
  const uploading = items.some((a) => a.progress !== undefined && a.progress < 100 && !a.error);
  useEffect(() => {
    if (!inView || !uploading) return;
    if (prefersReducedMotion()) return setItems((s) => s.map((a) => (a.error ? a : { ...a, progress: undefined })));
    const id = window.setInterval(() => setItems((s) => s.map((a) => (a.progress !== undefined && a.progress < 100 && !a.error ? { ...a, progress: Math.min(100, a.progress + 3 + (a.size % 5)) } : a))), 120);
    return () => window.clearInterval(id);
  }, [inView, uploading, setItems]);
  const limitFor = (k: MediaKind) => support.find((s) => s.kind === k)?.maxMb ?? maxSizeMb;
  const add = (list: Omit<MediaAttachment, 'id'>[]) => {
    setNotice('');
    setItems((s) => {
      const room = Math.max(0, maxFiles - s.length);
      if (list.length > room) setNotice(fill(t.tooMany, { n: maxFiles }));
      const next = list.slice(0, room).map((a) => {
        const max = limitFor(a.kind);
        return { ...a, id: `new-${++seq.current}`, error: a.error ?? (a.size > max * 1e6 ? fill(t.tooLarge, { mb: max }) : undefined), progress: a.progress ?? 0 };
      });
      return [...s, ...next];
    });
  };
  const addFiles = (files: FileList | File[]) =>
    add(
      Array.from(files).map((file) => {
        const kind = kindOf(file);
        const url = kind === 'image' && typeof URL !== 'undefined' ? URL.createObjectURL(file) : null;
        return { name: file.name, kind: kind ?? 'doc', size: file.size, error: kind ? undefined : t.unsupported, media: url ? <img src={url} alt="" /> : undefined };
      }),
    );
  const remove = (id: string) => (setItems((s) => s.filter((a) => a.id !== id)), setNotice(''));
  return { items, setItems, add, addFiles, remove, notice, ref };
}

function useSent() {
  const [sent, setSent] = useState<string | null>(null);
  useEffect(() => {
    if (!sent) return;
    const id = window.setTimeout(() => setSent(null), 2600);
    return () => window.clearTimeout(id);
  }, [sent]);
  return [sent, setSent] as const;
}

function Stage({ title, description, children, className, wide }: { title?: string; description?: string; children: React.ReactNode; className?: string; wide?: boolean }) {
  return (
    <section className={cn('kit-glow relative overflow-clip bg-surface py-12 sm:py-16 lg:py-20', className)}>
      <Style />
      <div className="kit-container">
        <div className={cn('mx-auto w-full', wide ? 'max-w-[60rem]' : 'max-w-[46rem]')}>
          {(title || description) && (
            <Reveal className="mb-6 sm:mb-8">
              {title && <h2 className="text-balance text-[22px] font-semibold leading-tight tracking-[-0.02em] text-foreground sm:text-[26px]">{title}</h2>}
              {description && <p className="mt-2 max-w-[36rem] text-pretty text-[15px] leading-relaxed text-muted-foreground">{description}</p>}
            </Reveal>
          )}
          {children}
        </div>
      </div>
    </section>
  );
}

function Note({ children, icon = 'shield' }: { children: React.ReactNode; icon?: IconName }) {
  return (
    <p className="mt-3 flex items-start gap-1.5 px-1 text-[12.5px] leading-snug text-muted-foreground">
      <Icon name={icon} size={14} className="mt-px shrink-0" />
      <span>{children}</span>
    </p>
  );
}

/** The text field with chips above and a toolbar below */
function Composer({ t, text, setText, items, remove, tools, onSend, sent, busy, limitLine, dropping }: { t: Texts; text: string; setText: (v: string) => void; items: MediaAttachment[]; remove: (id: string) => void; tools?: React.ReactNode; onSend: () => void; sent: string | null; busy?: boolean; limitLine?: React.ReactNode; dropping?: boolean }) {
  const uid = useId();
  const row = useRef<HTMLUListElement>(null);
  const [fade, setFade] = useState('');
  // The edge fades only where more chips follow, measured again when chips come or go
  useEffect(() => {
    const el = row.current;
    if (!el) return setFade('');
    const update = () => {
      const max = el.scrollWidth - el.clientWidth;
      setFade(max <= 1 ? '' : el.scrollLeft <= 1 ? 'kit-fade-end' : el.scrollLeft >= max - 1 ? 'kit-fade-start' : 'kit-fade-x');
    };
    update();
    el.addEventListener('scroll', update, { passive: true });
    window.addEventListener('resize', update);
    return () => (el.removeEventListener('scroll', update), window.removeEventListener('resize', update));
  }, [items.length]);
  const uploading = items.some((a) => a.progress !== undefined && a.progress < 100 && !a.error);
  const ready = items.filter((a) => !a.error);
  return (
    <form
      onSubmit={(e) => {
        e.preventDefault();
        onSend();
      }}
      className={cn(card, 'relative p-2 transition-[border-color,box-shadow] duration-200 focus-within:border-[color-mix(in_oklab,var(--primary)_50%,var(--border))] focus-within:shadow-[var(--highlight),var(--shadow-md),0_0_0_3px_color-mix(in_oklab,var(--primary)_16%,transparent)]', dropping && 'border-primary')}
    >
      {items.length > 0 && (
        <ul ref={row} aria-label={t.attachments} className={cn('kit-no-scrollbar relative flex gap-2 overflow-x-auto p-1 pb-2 sm:flex-wrap sm:overflow-visible sm:[mask-image:none]', fade)}>
          {items.map((a, i) => (
            <Chip key={a.id} a={a} t={t} onRemove={() => remove(a.id)} delay={i * 50} />
          ))}
        </ul>
      )}
      <label htmlFor={`${uid}-m`} className="sr-only">
        {t.message}
      </label>
      <textarea
        id={`${uid}-m`}
        rows={2}
        value={text}
        onChange={(e) => setText(e.target.value)}
        onKeyDown={(e) => {
          if (e.key === 'Enter' && !e.shiftKey) {
            e.preventDefault();
            onSend();
          }
        }}
        placeholder={t.placeholder}
        className="block min-h-11 w-full resize-none bg-transparent px-2.5 py-2 text-[15px] leading-relaxed text-foreground [field-sizing:content] placeholder:text-subtle-foreground focus-visible:outline-none"
      />
      <div className="flex items-center gap-1 px-0.5 pt-1">
        {tools}
        <span className="min-w-0 flex-1 truncate px-1 text-right text-[11.5px] tabular-nums text-subtle-foreground max-sm:hidden">{limitLine}</span>
        <button type="submit" disabled={busy || uploading || (!text.trim() && !ready.length)} aria-label={t.send} className={cn(primaryBtn, 'max-sm:ml-auto max-sm:w-11 max-sm:px-0')}>
          <Icon name="send" size={15} />
          <span className="max-sm:sr-only">{t.send}</span>
        </button>
      </div>
      <p role="status" className="px-2 text-[12.5px] text-success-foreground empty:hidden">
        {sent && (
          <span className="mmi-up mt-1 inline-flex items-center gap-1.5 pb-1">
            <Icon name="check-circle" size={14} />
            {sent}
          </span>
        )}
      </p>
    </form>
  );
}

const sentLine = (t: Texts, n: number) => (n === 0 ? t.sentText : n === 1 ? t.sent1 : fill(t.sent, { n }));

/* ------------------------------------------------------------------ */
/* dropzone                                                            */
/* ------------------------------------------------------------------ */

function Dropzone({ t, title, description, defaultValue = '', attachments = [], maxFiles = 8, maxSizeMb = 20, support = [], note, onSubmit, className }: P) {
  const f = useFormat();
  const uid = useId();
  const files = useAttachments(attachments, maxFiles, maxSizeMb, support, t);
  const [text, setText] = useState(defaultValue);
  const [drag, setDrag] = useState(0);
  const [sent, setSent] = useSent();
  const input = useRef<HTMLInputElement>(null);
  const camera = useRef<HTMLInputElement>(null);
  const accept = 'image/*,application/pdf,audio/*,video/*,.doc,.docx,.txt,.md,.csv,.xls,.xlsx';
  const send = () => {
    const ready = files.items.filter((a) => !a.error);
    if (!text.trim() && !ready.length) return;
    onSubmit?.({ text, attachments: ready });
    setSent(sentLine(t, ready.length));
    setText('');
    files.setItems([]);
  };
  const drop = {
    onDragEnter: (e: React.DragEvent) => (e.preventDefault(), setDrag((n) => n + 1)),
    onDragOver: (e: React.DragEvent) => e.preventDefault(),
    onDragLeave: () => setDrag((n) => Math.max(0, n - 1)),
    onDrop: (e: React.DragEvent) => {
      e.preventDefault();
      setDrag(0);
      if (e.dataTransfer.files.length) files.addFiles(e.dataTransfer.files);
    },
  };
  const used = files.items.filter((a) => !a.error).reduce((s, a) => s + a.size, 0);

  return (
    <Stage title={title} description={description} className={className}>
      <div ref={files.ref} {...drop} className="relative">
        <input ref={input} id={`${uid}-f`} type="file" multiple accept={accept} className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
        <input ref={camera} type="file" accept="image/*" capture="environment" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />

        {/* The zone: a large target from tablets on, two buttons on phones */}
        <Reveal delay={40}>
          <div className={cn('relative mb-3 hidden overflow-hidden rounded-2xl border-[1.5px] border-dashed p-6 text-center transition-colors duration-200 sm:block', drag ? 'border-primary bg-[color-mix(in_oklab,var(--primary)_8%,var(--card))]' : 'border-border-strong bg-[color-mix(in_oklab,var(--card)_60%,transparent)]')}>
            <span aria-hidden className={cn('mx-auto grid size-12 place-items-center rounded-2xl border border-border bg-card text-foreground shadow-[var(--shadow-sm)] transition-transform duration-300 motion-reduce:transition-none', drag && '-translate-y-1 scale-105')}>
              <Icon name={drag ? 'download' : 'cloud-upload'} size={22} />
            </span>
            <p className="mt-3 text-[15px] font-medium text-foreground">{drag ? t.drop : t.dropHint}</p>
            <p className="mt-1 text-[12.5px] tabular-nums text-muted-foreground">{fill(t.limits, { count: files.items.length, max: maxFiles, mb: maxSizeMb })}</p>
            <button type="button" onClick={() => input.current?.click()} className={cn(quietBtn, 'mt-4')}>
              <Icon name="paperclip" size={15} />
              {t.choose}
            </button>
            {support.length > 0 && (
              <ul aria-label={t.reads} className="mx-auto mt-5 grid max-w-[38rem] grid-cols-2 gap-2 text-left md:grid-cols-4">
                {support.map((s) => (
                  <li key={s.kind} className="rounded-xl border border-border bg-card p-3">
                    <p className="flex items-center gap-1.5 text-[12.5px] font-medium text-foreground">
                      <Icon name={KIND[s.kind].icon} size={14} style={{ color: `color-mix(in oklab, var(${KIND[s.kind].tone}) 75%, var(--foreground))` }} />
                      {t[s.kind]}
                    </p>
                    <p className="mt-1 text-[11.5px] leading-snug text-muted-foreground">{s.reads}</p>
                    <p className="mt-1.5 text-[11.5px] tabular-nums text-subtle-foreground">{[s.maxMb ? `${s.maxMb} MB` : null, s.maxSeconds ? fill(t.upTo, { v: f.time(s.maxSeconds) }) : null].filter(Boolean).join(' · ')}</p>
                  </li>
                ))}
              </ul>
            )}
          </div>
          <div className="mb-3 grid grid-cols-2 gap-2 sm:hidden">
            <button type="button" onClick={() => camera.current?.click()} className={cn(quietBtn, 'h-12')}>
              <Icon name="camera" size={16} />
              {t.takePhoto}
            </button>
            <button type="button" onClick={() => input.current?.click()} className={cn(quietBtn, 'h-12')}>
              <Icon name="paperclip" size={16} />
              {t.choose}
            </button>
          </div>
        </Reveal>

        <Reveal delay={100}>
          <Composer
            t={t}
            text={text}
            setText={setText}
            items={files.items}
            remove={files.remove}
            onSend={send}
            sent={sent}
            dropping={drag > 0}
            limitLine={fill(t.total, { used: f.mb(used), max: maxSizeMb * maxFiles })}
            tools={
              <button type="button" aria-label={t.addFile} title={t.addFile} onClick={() => input.current?.click()} className={toolBtn}>
                <Icon name="paperclip" size={18} />
              </button>
            }
          />
        </Reveal>
        {files.notice && (
          <p role="alert" className="mmi-up mt-2 flex items-start gap-1.5 px-1 text-[12.5px] text-destructive">
            <Icon name="alert-circle" size={14} className="mt-px shrink-0" />
            {files.notice}
          </p>
        )}
        <p className="mt-2 px-1 text-[12px] tabular-nums text-muted-foreground sm:hidden">{fill(t.limits, { count: files.items.length, max: maxFiles, mb: maxSizeMb })}</p>
        {note && <Note>{note}</Note>}
      </div>
    </Stage>
  );
}

/* ------------------------------------------------------------------ */
/* media-toolbar                                                       */
/* ------------------------------------------------------------------ */

function MediaToolbar({ t, title, description, defaultValue = '', attachments = [], maxFiles = 8, maxSizeMb = 20, support = [], screen, note, onSubmit, className }: P) {
  const f = useFormat();
  const files = useAttachments(attachments, maxFiles, maxSizeMb, support, t);
  const [text, setText] = useState(defaultValue);
  const [sent, setSent] = useSent();
  const [rec, setRec] = useState<number | null>(null);
  const [flash, setFlash] = useState(0);
  const image = useRef<HTMLInputElement>(null);
  const file = useRef<HTMLInputElement>(null);
  const camera = useRef<HTMLInputElement>(null);
  const shots = useRef(0);
  const total = maxSizeMb * maxFiles;
  const used = files.items.filter((a) => !a.error).reduce((s, a) => s + a.size, 0);
  const share = Math.min(100, (used / (total * 1e6)) * 100);
  const supported = (k: MediaKind) => !support.length || support.some((s) => s.kind === k);

  useEffect(() => {
    if (rec === null || prefersReducedMotion()) return;
    const id = window.setInterval(() => setRec((r) => (r === null ? r : r + 1)), 1000);
    return () => window.clearInterval(id);
  }, [rec === null]); // eslint-disable-line react-hooks/exhaustive-deps

  const tools: { id: string; icon: IconName; label: string; kind: MediaKind; run: () => void }[] = [
    { id: 'image', icon: 'image', label: t.addImage, kind: 'image', run: () => image.current?.click() },
    { id: 'file', icon: 'paperclip', label: t.addFile, kind: 'pdf', run: () => file.current?.click() },
    { id: 'camera', icon: 'camera', label: t.useCamera, kind: 'image', run: () => camera.current?.click() },
    {
      id: 'screen',
      icon: 'monitor',
      label: t.captureScreen,
      kind: 'screenshot',
      run: () => {
        setFlash((n) => n + 1);
        files.add([{ name: `${t.screenshot} ${++shots.current}.png`, kind: 'screenshot', size: 840_000 + shots.current * 61_000, media: screen }]);
      },
    },
    {
      id: 'voice',
      icon: 'mic',
      label: t.recordVoice,
      kind: 'audio',
      run: () => {
        if (rec === null) return setRec(0);
        files.add([{ name: `${t.voiceNote}.m4a`, kind: 'audio', size: 24_000 * Math.max(1, rec), duration: Math.max(1, rec) }]);
        setRec(null);
      },
    },
  ];
  const send = () => {
    const ready = files.items.filter((a) => !a.error);
    if (!text.trim() && !ready.length) return;
    onSubmit?.({ text, attachments: ready });
    setSent(sentLine(t, ready.length));
    setText('');
    files.setItems([]);
  };

  return (
    <Stage title={title} description={description} className={className}>
      <div ref={files.ref}>
        <input ref={image} type="file" multiple accept="image/*" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
        <input ref={file} type="file" multiple accept="application/pdf,.doc,.docx,.txt,.md,.csv,.xls,.xlsx,audio/*,video/*" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
        <input ref={camera} type="file" accept="image/*" capture="environment" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
        {flash > 0 && <span key={flash} aria-hidden className="mmi-flash pointer-events-none fixed inset-0 z-50 bg-white opacity-0" />}
        <Reveal delay={60}>
          <Composer
            t={t}
            text={text}
            setText={setText}
            items={files.items}
            remove={files.remove}
            onSend={send}
            sent={sent}
            busy={rec !== null}
            limitLine={fill(t.limits, { count: files.items.length, max: maxFiles, mb: maxSizeMb })}
            tools={
              <div role="group" aria-label={t.attachments} className="flex items-center gap-0.5">
                {tools.map((x) =>
                  x.id === 'voice' && rec !== null ? (
                    <button key={x.id} type="button" onClick={x.run} className={cn(btn, 'gap-2 bg-[color-mix(in_oklab,var(--destructive)_12%,transparent)] px-2.5 text-destructive hover:bg-[color-mix(in_oklab,var(--destructive)_18%,transparent)]')}>
                      <span aria-hidden className="mmi-blink size-2 rounded-full bg-destructive" />
                      <span className="tabular-nums">{f.time(rec)}</span>
                      <span className="sr-only">{t.stop}</span>
                      <Icon name="stop" size={13} />
                    </button>
                  ) : (
                    <button key={x.id} type="button" aria-label={x.label} title={x.label} disabled={!supported(x.kind) || rec !== null} onClick={x.run} className={cn(toolBtn, x.id === 'camera' && 'sm:hidden', x.id === 'screen' && 'max-sm:hidden')}>
                      <Icon name={x.icon} size={18} />
                    </button>
                  ),
                )}
              </div>
            }
          />
        </Reveal>
        {files.notice && (
          <p role="alert" className="mmi-up mt-2 flex items-start gap-1.5 px-1 text-[12.5px] text-destructive">
            <Icon name="alert-circle" size={14} className="mt-px shrink-0" />
            {files.notice}
          </p>
        )}

        {/* What the assistant reads, with the limits per kind */}
        <Reveal delay={140} className="mt-4 rounded-2xl border border-border bg-[color-mix(in_oklab,var(--card)_70%,transparent)] p-4">
          <div className="flex flex-wrap items-center justify-between gap-x-4 gap-y-2">
            <p className="text-[13px] font-medium text-foreground">{t.reads}</p>
            <div className="flex min-w-[12rem] items-center gap-2.5 max-sm:w-full">
              <span role="meter" aria-label={t.total.replace('{used}', f.mb(used)).replace('{max}', String(total))} aria-valuemin={0} aria-valuemax={100} aria-valuenow={Math.round(share)} className="block h-1.5 flex-1 overflow-hidden rounded-full bg-track">
                <span className={cn('block h-full origin-left rounded-full transition-transform duration-500', share > 85 ? 'bg-warning' : 'bg-primary')} style={{ transform: `scaleX(${round(share / 100, 3)})` }} />
              </span>
              <span className="text-[12px] tabular-nums text-muted-foreground">{fill(t.total, { used: f.mb(used), max: total })}</span>
            </div>
          </div>
          <ul className="mt-3 grid grid-cols-1 gap-x-6 gap-y-2.5 sm:grid-cols-2">
            {support.map((s) => (
              <li key={s.kind} className="flex items-start gap-2.5">
                <span aria-hidden className="grid size-8 shrink-0 place-items-center rounded-lg" style={{ background: `color-mix(in oklab, var(${KIND[s.kind].tone}) 14%, var(--card))`, color: `color-mix(in oklab, var(${KIND[s.kind].tone}) 75%, var(--foreground))` }}>
                  <Icon name={KIND[s.kind].icon} size={15} />
                </span>
                <span className="min-w-0">
                  <span className="flex flex-wrap items-baseline gap-x-2 text-[13px] font-medium text-foreground">
                    {t[s.kind]}
                    <span className="text-[11.5px] font-normal tabular-nums text-subtle-foreground">{[s.maxMb ? `${s.maxMb} MB` : null, s.maxSeconds ? fill(t.upTo, { v: f.time(s.maxSeconds) }) : null].filter(Boolean).join(' · ')}</span>
                  </span>
                  <span className="block text-[12.5px] leading-snug text-muted-foreground">{s.reads}</span>
                </span>
              </li>
            ))}
          </ul>
        </Reveal>
        {note && <Note>{note}</Note>}
      </div>
    </Stage>
  );
}

/* ------------------------------------------------------------------ */
/* voice-note                                                          */
/* ------------------------------------------------------------------ */

const BARS = 32;

/** A calm, believable input level: two slow waves and a little noise */
function levelAt(i: number, tick: number) {
  const v = 0.35 + 0.3 * Math.sin(tick / 3 + i / 2.4) + 0.22 * Math.sin(tick / 1.7 + i * 1.3) + 0.12 * Math.sin(tick * 1.9 + i * 3.1);
  return round(Math.max(0.08, Math.min(1, v)), 2);
}

function VoiceNote({ t, title, description, defaultValue = '', maxSeconds = 300, transcript = '', autoStart, note, onSubmit, className }: P) {
  const f = useFormat();
  const uid = useId();
  const [state, setState] = useState<'idle' | 'recording' | 'paused' | 'done'>('idle');
  const [secs, setSecs] = useState(0);
  const [tick, setTick] = useState(0);
  const [peaks, setPeaks] = useState<number[]>([]);
  const [mode, setMode] = useState<'audio' | 'text'>('audio');
  const [text, setText] = useState(defaultValue);
  const [sent, setSent] = useSent();
  const [ref, inView] = useInView<HTMLDivElement>();
  const started = useRef(false);
  const words = transcript.split(/(\s+)/);
  // About two words a second appear while recording, all of them once it stops
  const heard = state === 'idle' ? 0 : state === 'done' ? words.length : Math.min(words.length, secs * 4);

  useEffect(() => {
    if (autoStart && inView && !started.current && !prefersReducedMotion()) {
      started.current = true;
      setState('recording');
    }
  }, [autoStart, inView]);
  useEffect(() => {
    if (state !== 'recording') return;
    if (prefersReducedMotion()) {
      setSecs((s) => Math.max(s, 14));
      return;
    }
    const a = window.setInterval(() => setTick((n) => n + 1), 110);
    const b = window.setInterval(() => setSecs((s) => (s + 1 >= maxSeconds ? (setState('done'), maxSeconds) : s + 1)), 1000);
    return () => (window.clearInterval(a), window.clearInterval(b));
  }, [state, maxSeconds]);
  useEffect(() => {
    if (state === 'recording' && tick % 3 === 0) setPeaks((p) => [...p.slice(-47), levelAt(0, tick)]);
  }, [tick, state]);

  const reset = () => (setState('idle'), setSecs(0), setPeaks([]), setTick(0));
  const levels = Array.from({ length: BARS }, (_, i) => (state === 'recording' ? levelAt(i, tick) : state === 'paused' ? 0.1 : levelAt(i, 7) * 0.5));
  const share = Math.min(100, (secs / maxSeconds) * 100);
  const wave = peaks.length ? peaks : Array.from({ length: 48 }, (_, i) => levelAt(i, 4));
  const send = () => {
    const att: MediaAttachment[] = state === 'done' && mode === 'audio' ? [{ id: 'voice', name: `${t.voiceNote}.m4a`, kind: 'audio', size: secs * 24_000, duration: secs }] : [];
    const body = state === 'done' && mode === 'text' ? [text, transcript.replace(/~/g, '')].filter(Boolean).join(' ') : text;
    if (!body.trim() && !att.length) return;
    onSubmit?.({ text: body, attachments: att });
    setSent(sentLine(t, att.length));
    setText('');
    reset();
  };

  return (
    <Stage title={title} description={description} className={className}>
      <div ref={ref}>
        <Reveal effect="scale" className={cn(card, 'overflow-hidden p-0')}>
          {/* Recorder */}
          <div className={cn('relative px-4 pb-5 pt-6 text-center transition-colors duration-300 sm:px-6', state === 'recording' && 'bg-[color-mix(in_oklab,var(--destructive)_5%,var(--card))]')}>
            <p className="flex items-center justify-center gap-2 text-[13px] font-medium" aria-live="polite">
              {state === 'recording' && <span aria-hidden className="mmi-blink size-2 rounded-full bg-destructive" />}
              <span className={state === 'recording' ? 'text-destructive' : 'text-muted-foreground'}>{state === 'recording' ? t.recording : state === 'paused' ? t.paused : state === 'done' ? t.voiceNote : t.record}</span>
            </p>
            <p className="mt-1 text-[34px] font-semibold tabular-nums tracking-[-0.03em] text-foreground sm:text-[40px]">{f.time(secs)}</p>
            <p className="text-[12px] tabular-nums text-muted-foreground">{fill(t.left, { v: f.time(Math.max(0, maxSeconds - secs)) })}</p>

            {state === 'done' ? (
              <div className="mmi-up mx-auto mt-5 flex max-w-[28rem] items-center gap-3 rounded-xl border border-border bg-surface p-2.5 text-left">
                <span aria-hidden className="grid size-10 shrink-0 place-items-center rounded-lg bg-[color-mix(in_oklab,var(--warning)_15%,var(--card))] text-warning-foreground">
                  <Icon name="mic" size={17} />
                </span>
                <svg aria-hidden viewBox={`0 0 ${wave.length * 4} 32`} preserveAspectRatio="none" className="h-8 min-w-0 flex-1">
                  {wave.map((v, i) => (
                    <rect key={i} x={i * 4 + 0.75} y={round(16 - v * 14)} width="2.5" height={round(Math.max(2, v * 28))} rx="1.25" fill="color-mix(in oklab, var(--foreground) 55%, transparent)" />
                  ))}
                </svg>
                <span className="shrink-0 text-[12px] tabular-nums text-muted-foreground">{f.time(secs)}</span>
              </div>
            ) : (
              <div role="meter" aria-label={t.level} aria-valuemin={0} aria-valuemax={100} aria-valuenow={Math.round((levels.reduce((s, v) => s + v, 0) / BARS) * 100)} className="mx-auto mt-5 flex h-16 max-w-[28rem] items-center justify-center gap-[3px]">
                {levels.map((v, i) => (
                  <span key={i} aria-hidden className={cn('w-[5px] rounded-full transition-[height] duration-100 ease-linear motion-reduce:transition-none', state === 'recording' ? 'bg-destructive' : 'bg-[color-mix(in_oklab,var(--foreground)_22%,transparent)]')} style={{ height: `${round(8 + v * 56)}px`, opacity: state === 'recording' ? round(0.45 + v * 0.55, 2) : 1 }} />
                ))}
              </div>
            )}

            <div aria-hidden className="mx-auto mt-4 h-1 max-w-[28rem] overflow-hidden rounded-full bg-track">
              <span className="block h-full origin-left rounded-full bg-destructive transition-transform duration-1000 ease-linear" style={{ transform: `scaleX(${round(share / 100, 3)})` }} />
            </div>

            <div className="mt-5 flex flex-wrap items-center justify-center gap-2">
              {state === 'idle' && (
                <button type="button" onClick={() => setState('recording')} className="grid size-16 place-items-center rounded-full bg-destructive text-destructive-foreground shadow-[0_6px_20px_color-mix(in_oklab,var(--destructive)_35%,transparent)] transition-transform hover:scale-105 active:scale-95 motion-reduce:transition-none" aria-label={t.record}>
                  <Icon name="mic" size={24} />
                </button>
              )}
              {(state === 'recording' || state === 'paused') && (
                <>
                  <button type="button" onClick={reset} className={quietBtn}>
                    <Icon name="trash" size={15} />
                    {t.discard}
                  </button>
                  <button type="button" onClick={() => setState(state === 'recording' ? 'paused' : 'recording')} className={quietBtn}>
                    <Icon name={state === 'recording' ? 'pause' : 'play'} size={15} />
                    {state === 'recording' ? t.pause : t.resume}
                  </button>
                  <button type="button" onClick={() => setState('done')} className={primaryBtn}>
                    <Icon name="stop" size={14} />
                    {t.stop}
                  </button>
                </>
              )}
              {state === 'done' && (
                <>
                  <button type="button" onClick={reset} className={quietBtn}>
                    <Icon name="trash" size={15} />
                    {t.discard}
                  </button>
                  <div role="radiogroup" aria-label={t.voiceNote} className="flex gap-1 rounded-lg bg-[color-mix(in_oklab,var(--foreground)_6%,transparent)] p-1">
                    {(['audio', 'text'] as const).map((m) => (
                      <button key={m} type="button" role="radio" aria-checked={mode === m} tabIndex={mode === m ? 0 : -1} onClick={() => setMode(m)} onKeyDown={(e) => ['ArrowLeft', 'ArrowRight'].includes(e.key) && (e.preventDefault(), setMode(m === 'audio' ? 'text' : 'audio'), (e.currentTarget.parentElement?.querySelector(`[data-m="${m === 'audio' ? 'text' : 'audio'}"]`) as HTMLElement | null)?.focus())} data-m={m} className={cn('inline-flex h-11 items-center gap-1.5 rounded-md px-3 text-[12.5px] font-medium transition-colors sm:h-8', mode === m ? 'bg-card text-foreground shadow-[var(--highlight),var(--shadow-sm)]' : 'text-muted-foreground hover:text-foreground')}>
                        <Icon name={m === 'audio' ? 'volume' : 'type'} size={14} />
                        {m === 'audio' ? t.asAudio : t.asText}
                      </button>
                    ))}
                  </div>
                </>
              )}
            </div>
          </div>

          {/* Transcript as it is heard */}
          {transcript && state !== 'idle' && (
            <div className="border-t border-border px-4 py-4 sm:px-6">
              <p className="text-[12px] font-medium text-muted-foreground">{t.transcript}</p>
              <p className="mt-1.5 text-pretty text-[14.5px] leading-relaxed text-foreground" aria-live="polite">
                {words.slice(0, heard).map((w, i) =>
                  w.startsWith('~') ? (
                    <span key={i} className="underline decoration-[color-mix(in_oklab,var(--warning)_80%,transparent)] decoration-dotted decoration-2 underline-offset-4">
                      {w.replace(/~/g, '')}
                    </span>
                  ) : (
                    <span key={i}>{w.replace(/~/g, '')}</span>
                  ),
                )}
                {state === 'recording' && <span aria-hidden className="mmi-blink ml-0.5 inline-block h-4 w-0.5 translate-y-0.5 bg-foreground" />}
              </p>
              {transcript.includes('~') && <p className="mt-2 text-[12px] text-muted-foreground">{t.uncertain}</p>}
            </div>
          )}

          {/* Message */}
          <form
            onSubmit={(e) => {
              e.preventDefault();
              send();
            }}
            className="flex items-end gap-2 border-t border-border p-2"
          >
            <label htmlFor={`${uid}-m`} className="sr-only">
              {t.message}
            </label>
            <textarea id={`${uid}-m`} rows={1} value={text} onChange={(e) => setText(e.target.value)} placeholder={t.placeholder} className="block min-h-11 w-0 min-w-0 flex-1 resize-none bg-transparent px-2.5 py-2.5 text-[15px] leading-snug text-foreground [field-sizing:content] placeholder:text-subtle-foreground focus-visible:outline-none" />
            <button type="submit" disabled={state === 'recording' || state === 'paused' || (!text.trim() && state !== 'done')} aria-label={t.send} className={cn(primaryBtn, 'max-sm:w-11 max-sm:px-0')}>
              <Icon name="send" size={15} />
              <span className="max-sm:sr-only">{t.send}</span>
            </button>
          </form>
          <p role="status" className="px-4 pb-3 text-[12.5px] text-success-foreground empty:hidden">
            {sent && (
              <span className="mmi-up inline-flex items-center gap-1.5">
                <Icon name="check-circle" size={14} />
                {sent}
              </span>
            )}
          </p>
        </Reveal>
        <Note icon="mic">{note ?? t.micNote}</Note>
      </div>
    </Stage>
  );
}

/* ------------------------------------------------------------------ */
/* camera-capture                                                      */
/* ------------------------------------------------------------------ */

type Box = { x: number; y: number; w: number; h: number };

function CameraCapture({ t, title, description, defaultValue = '', preview, screen, windows = [], maxFiles = 8, maxSizeMb = 20, support = [], note, onSubmit, className }: P) {
  const uid = useId();
  const files = useAttachments([], maxFiles, maxSizeMb, support, t);
  const [mode, setMode] = useState<'photo' | 'screen'>('photo');
  const [shot, setShot] = useState(false);
  const [flash, setFlash] = useState(0);
  const [box, setBox] = useState<Box | null>(null);
  const [win, setWin] = useState(windows[0]?.id ?? '');
  const [text, setText] = useState(defaultValue);
  const [sent, setSent] = useSent();
  const [front, setFront] = useState(false);
  const frame = useRef<HTMLDivElement>(null);
  const start = useRef<{ x: number; y: number } | null>(null);
  const seq = useRef(0);
  const pic = mode === 'photo' ? preview : screen;
  const pct = (e: React.PointerEvent) => {
    const r = frame.current!.getBoundingClientRect();
    return { x: round(Math.max(0, Math.min(100, ((e.clientX - r.left) / r.width) * 100)), 1), y: round(Math.max(0, Math.min(100, ((e.clientY - r.top) / r.height) * 100)), 1) };
  };
  const capture = () => {
    setFlash((n) => n + 1);
    setShot(true);
    setBox(null);
  };
  const attach = () => {
    const name = mode === 'photo' ? `${t.photo} ${++seq.current}.jpg` : `${windows.find((w) => w.id === win)?.label ?? t.screen}.png`;
    files.add([{ name: box ? `${name.replace(/\.\w+$/, '')} · ${t.area}${mode === 'photo' ? '.jpg' : '.png'}` : name, kind: mode === 'photo' ? 'image' : 'screenshot', size: mode === 'photo' ? 2_400_000 : 910_000, media: pic }]);
    setShot(false);
    setBox(null);
  };
  const send = () => {
    const ready = files.items.filter((a) => !a.error);
    if (!text.trim() && !ready.length) return;
    onSubmit?.({ text, attachments: ready });
    setSent(sentLine(t, ready.length));
    setText('');
    files.setItems([]);
  };
  const modes = [
    { id: 'photo' as const, label: t.camera, icon: 'camera' as IconName },
    { id: 'screen' as const, label: t.screen, icon: 'monitor' as IconName },
  ];

  return (
    <Stage title={title} description={description} className={className} wide>
      <div ref={files.ref} className="grid grid-cols-1 gap-4 lg:grid-cols-[minmax(0,1fr)_22rem] lg:items-start lg:gap-6">
        <Reveal effect="scale" className="min-w-0">
          <div className="dark rounded-2xl border border-border bg-bezel p-2 shadow-[var(--shadow-lg)]">
            <div className="flex items-center justify-between gap-2 px-1 pb-2">
              <div role="radiogroup" aria-label={t.capture} className="flex gap-1 rounded-lg bg-[rgb(255_255_255/0.08)] p-1">
                {modes.map((m) => (
                  <button key={m.id} type="button" role="radio" aria-checked={mode === m.id} tabIndex={mode === m.id ? 0 : -1} data-m={m.id} onClick={() => (setMode(m.id), setShot(false), setBox(null))} onKeyDown={(e) => ['ArrowLeft', 'ArrowRight'].includes(e.key) && (e.preventDefault(), setMode(m.id === 'photo' ? 'screen' : 'photo'), setShot(false), setBox(null), (e.currentTarget.parentElement?.querySelector(`[data-m="${m.id === 'photo' ? 'screen' : 'photo'}"]`) as HTMLElement | null)?.focus())} className={cn('inline-flex h-11 items-center gap-1.5 rounded-md px-3 text-[12.5px] font-medium transition-colors sm:h-8', mode === m.id ? 'bg-white text-[#111]' : 'text-white/75 hover:text-white')}>
                    <Icon name={m.icon} size={14} />
                    {m.label}
                  </button>
                ))}
              </div>
              {mode === 'photo' && !shot && (
                <button type="button" aria-label={t.flip} title={t.flip} onClick={() => setFront((v) => !v)} className="grid size-11 place-items-center rounded-lg text-white/80 hover:bg-white/10 hover:text-white sm:size-9">
                  <Icon name="refresh" size={17} />
                </button>
              )}
            </div>

            {mode === 'screen' && !shot && windows.length > 0 && (
              <div
                role="radiogroup"
                aria-label={t.pickWindow}
                // APG radio group: one tab stop, the arrow keys move and choose
                onKeyDown={(e) => {
                  const step = e.key === 'ArrowRight' || e.key === 'ArrowDown' ? 1 : e.key === 'ArrowLeft' || e.key === 'ArrowUp' ? -1 : 0;
                  if (!step) return;
                  e.preventDefault();
                  const i = windows.findIndex((w) => w.id === win);
                  const next = windows[(i + step + windows.length) % windows.length];
                  setWin(next.id);
                  (e.currentTarget.querySelector('[data-w="' + next.id + '"]') as HTMLElement | null)?.focus();
                }}
                className="kit-no-scrollbar mb-2 flex gap-1.5 overflow-x-auto px-1"
              >
                {windows.map((w) => (
                  <button key={w.id} type="button" role="radio" aria-checked={win === w.id} tabIndex={win === w.id ? 0 : -1} data-w={w.id} onClick={() => setWin(w.id)} className={cn('inline-flex h-11 shrink-0 items-center gap-1.5 rounded-lg border px-3 text-[12.5px] font-medium transition-colors sm:h-9', win === w.id ? 'border-white/70 bg-white/12 text-white' : 'border-white/15 text-white/70 hover:text-white')}>
                    <Icon name={w.icon ?? 'layout'} size={14} />
                    {w.label}
                  </button>
                ))}
              </div>
            )}

            <div
              ref={frame}
              role={shot ? 'group' : 'img'}
              aria-label={shot ? t.markArea : t.viewfinder}
              className={cn('relative isolate aspect-[4/3] overflow-hidden rounded-xl bg-black', shot && 'cursor-crosshair touch-none select-none')}
              onPointerDown={(e) => {
                if (!shot || e.button !== 0) return;
                e.currentTarget.setPointerCapture(e.pointerId);
                start.current = pct(e);
                setBox({ ...start.current, w: 0, h: 0 });
              }}
              onPointerMove={(e) => {
                if (!start.current) return;
                const p = pct(e);
                const s = start.current;
                setBox({ x: Math.min(s.x, p.x), y: Math.min(s.y, p.y), w: round(Math.abs(p.x - s.x), 1), h: round(Math.abs(p.y - s.y), 1) });
              }}
              onPointerUp={() => {
                start.current = null;
                setBox((b) => (b && b.w > 3 && b.h > 3 ? b : null));
              }}
            >
              <div data-clip-ok className={cn('absolute inset-0 [&>*]:size-full', mode === 'photo' && front && !shot && '-scale-x-100')}>
                {pic}
              </div>
              {!shot && mode === 'photo' && (
                <div aria-hidden className="pointer-events-none absolute inset-6">
                  {['left-0 top-0 border-l-2 border-t-2 rounded-tl-lg', 'right-0 top-0 border-r-2 border-t-2 rounded-tr-lg', 'bottom-0 left-0 border-b-2 border-l-2 rounded-bl-lg', 'bottom-0 right-0 border-b-2 border-r-2 rounded-br-lg'].map((c) => (
                    <span key={c} className={cn('absolute size-7 border-white/85', c)} />
                  ))}
                </div>
              )}
              {box && (
                <>
                  <div aria-hidden className="pointer-events-none absolute inset-0 bg-black/45" style={{ clipPath: `polygon(0 0,100% 0,100% 100%,0 100%,0 0,${box.x}% ${box.y}%,${box.x}% ${box.y + box.h}%,${box.x + box.w}% ${box.y + box.h}%,${box.x + box.w}% ${box.y}%,${box.x}% ${box.y}%)`, clipRule: 'evenodd' }} />
                  <div aria-hidden className="pointer-events-none absolute rounded-md border-2 border-white shadow-[0_0_0_1px_rgb(0_0_0/0.4)]" style={{ left: `${box.x}%`, top: `${box.y}%`, width: `${box.w}%`, height: `${box.h}%` }} />
                </>
              )}
              {shot && !box && (
                <span className="pointer-events-none absolute inset-x-0 bottom-3 mx-auto w-fit max-w-[90%] rounded-lg bg-black/65 px-3 py-1.5 text-center text-[12.5px] font-medium text-white backdrop-blur-sm">
                  <Icon name="crop" size={13} className="mr-1.5 inline align-[-2px]" />
                  {t.markArea}
                </span>
              )}
              {flash > 0 && <span key={flash} aria-hidden className="mmi-flash pointer-events-none absolute inset-0 bg-white opacity-0" />}
            </div>

            <div className="flex items-center justify-center gap-3 px-1 pb-1 pt-3">
              {shot ? (
                <>
                  <button type="button" onClick={() => (setShot(false), setBox(null))} className="inline-flex h-11 items-center gap-1.5 rounded-lg border border-white/20 px-3.5 text-[13px] font-medium text-white hover:bg-white/10">
                    <Icon name="rotate-ccw" size={15} />
                    {t.retake}
                  </button>
                  {box && (
                    <button type="button" onClick={() => setBox(null)} className="inline-flex h-11 items-center gap-1.5 rounded-lg border border-white/20 px-3.5 text-[13px] font-medium text-white hover:bg-white/10">
                      {t.wholePicture}
                    </button>
                  )}
                  <button type="button" onClick={attach} className="inline-flex h-11 items-center gap-1.5 rounded-lg bg-white px-4 text-[13px] font-semibold text-[#111] hover:bg-white/90">
                    <Icon name="paperclip" size={15} />
                    {t.attach}
                  </button>
                </>
              ) : (
                <button type="button" aria-label={mode === 'photo' ? t.shutter : t.capture} onClick={capture} className="grid size-16 place-items-center rounded-full border-4 border-white/85 transition-transform hover:scale-105 active:scale-95 motion-reduce:transition-none">
                  <span className="size-11 rounded-full bg-white" />
                </button>
              )}
            </div>
          </div>
        </Reveal>

        <Reveal delay={120} className="lg:sticky lg:top-6">
          <Composer
            t={t}
            text={text}
            setText={setText}
            items={files.items}
            remove={files.remove}
            onSend={send}
            sent={sent}
            limitLine={undefined}
            tools={
              <span className="inline-flex items-center gap-1.5 px-1.5 text-[12px] tabular-nums text-muted-foreground" id={`${uid}-n`}>
                <Icon name="paperclip" size={14} />
                {files.items.length}/{maxFiles}
              </span>
            }
          />
          <Note>{note ?? t.cameraNote}</Note>
          <ol className="mt-5 space-y-3 border-t border-border pt-5 max-lg:hidden">
            {t.howTo.split('|').map((x, i) => (
              <li key={x} className="flex items-center gap-3 text-[13.5px] text-foreground">
                <span aria-hidden className="grid size-7 shrink-0 place-items-center rounded-full border border-border bg-card text-[12px] font-semibold tabular-nums text-muted-foreground">{i + 1}</span>
                {x}
              </li>
            ))}
          </ol>
        </Reveal>
      </div>
    </Stage>
  );
}

/* ------------------------------------------------------------------ */

/**
 * Input for an AI that reads more than text: a drop zone with the limits per
 * kind, a toolbar for image, file, camera, screenshot and voice with what the
 * assistant reads, a voice note with live level, timer and transcript, and a
 * camera or screen capture where you mark the part that matters. Chips show
 * upload progress, size and errors. Text features (commands, mentions) are
 * prompt-input.
 */
export function MultimodalInput({ variant = 'dropzone', labels, ...props }: MultimodalInputProps) {
  const t = useTexts(TEXTS, labels);
  const p: P = { ...props, t };
  switch (variant) {
    case 'media-toolbar':
      return <MediaToolbar {...p} />;
    case 'voice-note':
      return <VoiceNote {...p} />;
    case 'camera-capture':
      return <CameraCapture {...p} />;
    default:
      return <Dropzone {...p} />;
  }
}

Frequent questions

How do I install Multimodal input?

With aiw add @ai-website/multimodal-input. The command puts components/ui/multimodal-input.tsx into your project, the code is yours from then on. It brings along icon and reveal. It needs no further npm packages.

Which variants does Multimodal input have?

4 variants, Drop zone, Media toolbar, Voice note and Camera and screen. You pick one with the variant prop, for example variant="media-toolbar".

Can I translate the texts of Multimodal input?

Yes. Multimodal input carries 67 texts of its own in English and German, such as “Message”, “Ask about your files, photos or a recording” and “Send”. The language follows the KitProvider, single texts change through the labels prop.

Can an AI assistant build with Multimodal input?

Yes. Through the MCP server of the aiw CLI, Claude, Cursor, GitHub Copilot, Windsurf, Codex and Gemini find Multimodal input with aiw_components and install it with aiw_add @ai-website/multimodal-input. Name the variant in your prompt, for example variant="dropzone", and the assistant writes your content straight into the props.

Often searched

Jump to

Copied to the clipboard