Multimodal AI input
Input for an AI that reads more than text. A drop zone with limits per kind, a toolbar for image, file, camera, screenshot and voice, a voice note with level, timer and transcript, and a camera or screen capture where you mark the part that matters. Text features like commands and mentions live in prompt-input.
Show the assistant, not only tell it
Many questions are easier to show than to type, a screenshot of an error, a photo of a receipt, a voice note on the way. Multimodal input lets an AI assistant take all of that and says up front what it will read, so nobody uploads a file in vain.
Drop zone is a large target for files from tablets on, with what each kind may weigh and what gets read, chips show progress, size and why a file is refused, and phones get two buttons for photo and files. Media toolbar puts one button per kind of media in the field, a screenshot lands as a chip with a flash and the voice button records right in the bar. Voice note records with live level, timer and time left, then sends as audio or inserts as text, the transcript appears while you speak and hard to hear words are marked. Camera and screen opens a viewfinder with guides or a window to capture, then you drag over the part that matters and attach it.
State the limits before the upload, not in an error after it. Let people crop a screenshot to the part that matters, the answer gets better and the request smaller. Text features like slash commands and mentions live in Prompt input, the two work side by side.
Variants
Details
What Multimodal input brings along, read from the registry and the source.
- Category
- AI and chat
- Area
- App
- Variants
- 4
- Version
- 1.0.0 · since 1.2.0
- File
- components/ui/multimodal-input.tsx
- Brings along
- utilskiticonreveal
- npm packages
- None
- Languages
- English and German, 67 texts of its own
- Studio
- Adjust in code
- In collections
- AI product
- Build with AI
- Over MCP in Claude, Cursor, Copilot and Codex
Props
The MultimodalInputProps interface from components/ui/multimodal-input.tsx, with the comments from the code.
variantoptionalMultimodalInputVariantdropzonemedia-toolbarvoice-notecamera-capture
titleoptionalstringdescriptionoptionalstringplaceholderoptionalstringdefaultValueoptionalstringText already in the field
attachmentsoptionalMediaAttachment[]maxFilesoptionalnumberMost attachments per message
maxSizeMboptionalnumberLargest file in MB, unless a kind sets its own
supportoptionalMediaSupport[]What the model reads per kind
maxSecondsoptionalnumbervoice-note: longest recording in seconds
transcriptoptionalstringvoice-note: the text heard, shown as the transcript
autoStartoptionalbooleanvoice-note: start recording once in view
previewoptionalReact.ReactNodecamera-capture: the picture the camera sees
screenoptionalReact.ReactNodecamera-capture: the picture a screenshot shows
windowsoptionalCaptureWindow[]camera-capture: windows to choose in screenshot mode
noteoptionalstringLine under the field, e.g. what happens with files and audio
onSubmitoptional(value: MultimodalSubmit) => void
Built-in texts
Multimodal input writes these texts itself. The language follows the KitProvider, the labels prop replaces single ones.
Show all 67 texts
| Key | English | German |
|---|---|---|
| message | English Message | German Nachricht |
| placeholder | English Ask about your files, photos or a recording | German Frag zu deinen Dateien, Fotos oder einer Aufnahme |
| send | English Send | German Senden |
| sent | English Sent with {n} attachments | German Gesendet mit {n} Anhängen |
| sent1 | English Sent with 1 attachment | German Gesendet mit 1 Anhang |
| sentText | English Sent | German Gesendet |
| attachments | English Attachments | German Anhänge |
| remove | English Remove {name} | German {name} entfernen |
| uploading | English Uploading, {n}% | German Wird hochgeladen, {n} % |
| ready | English Ready | German Bereit |
| tooLarge | English Larger than {mb} MB | German Größer als {mb} MB |
| tooLong | English Longer than {v} | German Länger als {v} |
| tooMany | English Up to {n} attachments per message. The rest was left out. | German Bis zu {n} Anhänge je Nachricht. Der Rest wurde weggelassen. |
| unsupported | English This file type cannot be read | German Diese Dateiart kann nicht gelesen werden |
| drop | English Drop to attach | German Zum Anhängen ablegen |
| dropHint | English Drop files here or choose them | German Dateien hier ablegen oder auswählen |
| choose | English Choose files | German Dateien wählen |
| takePhoto | English Take a photo | German Foto aufnehmen |
| limits | English {count} of {max} attachments · up to {mb} MB each | German {count} von {max} Anhängen · je bis {mb} MB |
| total | English {used} of {max} MB | German {used} von {max} MB |
| image | English Image | German Bild |
| screenshot | English Screenshot | German Bildschirmfoto |
| English PDF | German PDF | |
| doc | English Document | German Dokument |
| sheet | English Spreadsheet | German Tabelle |
| audio | English Audio | German Audio |
| video | English Video | German Video |
| file | English File | German Datei |
| camera | English Camera | German Kamera |
| screen | English Screen | German Bildschirm |
| voice | English Voice | German Sprache |
| addImage | English Add an image | German Bild hinzufügen |
| addFile | English Add a file | German Datei hinzufügen |
| useCamera | English Use the camera | German Kamera nutzen |
| captureScreen | English Take a screenshot | German Bildschirmfoto aufnehmen |
| recordVoice | English Record a voice note | German Sprachnachricht aufnehmen |
| reads | English What the assistant reads | German Was der Assistent liest |
| upTo | English up to {v} | German bis {v} |
| record | English Start recording | German Aufnahme starten |
| recording | English Recording | German Nimmt auf |
| paused | English Paused | German Pausiert |
| pause | English Pause | German Pausieren |
| resume | English Resume | German Weiter |
| stop | English Stop and keep | German Beenden und behalten |
| discard | English Discard | German Verwerfen |
| level | English Input level | German Eingangspegel |
| left | English {v} left | German noch {v} |
| voiceNote | English Voice note | German Sprachnachricht |
| asAudio | English Send as audio | German Als Audio senden |
| asText | English Insert as text | German Als Text einfügen |
| transcript | English Transcript | German Mitschrift |
| uncertain | English Words in dotted lines were hard to hear. | German Gepunktet unterstrichene Wörter waren schwer zu verstehen. |
| micNote | English The microphone runs only while the red light is on. | German Das Mikrofon läuft nur, solange der rote Punkt leuchtet. |
| photo | English Photo | German Foto |
| shutter | English Take picture | German Foto machen |
| flip | English Switch camera | German Kamera wechseln |
| retake | English Take again | German Neu aufnehmen |
| attach | English Attach | German Anhängen |
| attached | English Attached | German Angehängt |
| markArea | English Drag over the part the assistant should look at | German Ziehe über den Teil, den sich der Assistent ansehen soll |
| wholePicture | English Whole picture | German Ganzes Bild |
| area | English Marked area | German Markierter Bereich |
| pickWindow | English Choose what to capture | German Wähl, was aufgenommen wird |
| capture | English Capture | German Aufnehmen |
| cameraNote | English Pictures stay on this device until you send them. | German Bilder bleiben auf diesem Gerät, bis du sie sendest. |
| viewfinder | English Camera view | German Kamerabild |
| howTo | English Take a picture or capture a window|Drag over the part that matters|Attach it and ask your question | German Foto machen oder ein Fenster aufnehmen|Über den wichtigen Teil ziehen|Anhängen und deine Frage stellen |
Accessibility
Read from the source of the component.
- Multimodal input handles the keys Enter, Arrow up, Arrow down, Arrow left, Arrow right and Space itself.
- Sets aria-checked, aria-label, aria-live, aria-valuemax, aria-valuemin and aria-valuenow, so screen readers announce purpose and state.
- Uses the roles alert, group, meter, radio, radiogroup, shot and status.
- Announces changes through a live region without moving focus.
- Every field has a label, visible or for screen readers.
- Adds text for screen readers where icons and graphics stand alone.
- Shows a visible focus ring when used with the keyboard.
- The server renders the starting state in full, JavaScript only adds the interaction.
Motion
- Rises in briefly the first time it comes into view (kit-rise, useInView).
- Moves with the scroll, its progress follows the position on the page.
- Brings animations of its own, calmly timed and without wobble.
- With reduced motion (prefers-reduced-motion) it stays still. 4 places in the code switch motion off then.
Source
The file aiw add puts into your project. From then on it is yours.
Show the file, 1,160 lines
'use client';
import { useEffect, useId, useRef, useState } from 'react';
import { cn, prefersReducedMotion, round, useInView, useLocale, useTexts } from '@/lib/kit';
import { Icon, type IconName } from '@/components/ui/icon';
import { Reveal } from '@/components/ui/reveal';
export type MultimodalInputVariant = 'dropzone' | 'media-toolbar' | 'voice-note' | 'camera-capture';
export type MediaKind = 'image' | 'screenshot' | 'pdf' | 'doc' | 'sheet' | 'audio' | 'video';
export interface MediaAttachment {
id: string;
name: string;
kind: MediaKind;
/** Size in bytes */
size: number;
/** 0 to 100, below 100 the file is still uploading */
progress?: number;
/** A message when the file cannot be used */
error?: string;
/** Audio and video length in seconds */
duration?: number;
/** Preview for images and screenshots, a drawn motif or your own <img> */
media?: React.ReactNode;
}
/** What the model reads per kind of media, shown as plain words */
export interface MediaSupport {
kind: MediaKind;
/** e.g. "Reads text and tables" */
reads: string;
/** Largest file in MB */
maxMb?: number;
/** Longest audio or video in seconds */
maxSeconds?: number;
}
/** camera-capture: a window to share in screenshot mode */
export interface CaptureWindow {
id: string;
label: string;
icon?: IconName;
}
export interface MultimodalSubmit {
text: string;
attachments: MediaAttachment[];
}
export interface MultimodalInputProps {
variant?: MultimodalInputVariant;
title?: string;
description?: string;
placeholder?: string;
/** Text already in the field */
defaultValue?: string;
attachments?: MediaAttachment[];
/** Most attachments per message */
maxFiles?: number;
/** Largest file in MB, unless a kind sets its own */
maxSizeMb?: number;
/** What the model reads per kind */
support?: MediaSupport[];
/** voice-note: longest recording in seconds */
maxSeconds?: number;
/** voice-note: the text heard, shown as the transcript */
transcript?: string;
/** voice-note: start recording once in view */
autoStart?: boolean;
/** camera-capture: the picture the camera sees */
preview?: React.ReactNode;
/** camera-capture: the picture a screenshot shows */
screen?: React.ReactNode;
/** camera-capture: windows to choose in screenshot mode */
windows?: CaptureWindow[];
/** Line under the field, e.g. what happens with files and audio */
note?: string;
onSubmit?: (value: MultimodalSubmit) => void;
labels?: Partial<(typeof TEXTS)['en']>;
className?: string;
}
const TEXTS = {
en: {
message: 'Message',
placeholder: 'Ask about your files, photos or a recording',
send: 'Send',
sent: 'Sent with {n} attachments',
sent1: 'Sent with 1 attachment',
sentText: 'Sent',
attachments: 'Attachments',
remove: 'Remove {name}',
uploading: 'Uploading, {n}%',
ready: 'Ready',
tooLarge: 'Larger than {mb} MB',
tooLong: 'Longer than {v}',
tooMany: 'Up to {n} attachments per message. The rest was left out.',
unsupported: 'This file type cannot be read',
drop: 'Drop to attach',
dropHint: 'Drop files here or choose them',
choose: 'Choose files',
takePhoto: 'Take a photo',
limits: '{count} of {max} attachments · up to {mb} MB each',
total: '{used} of {max} MB',
image: 'Image',
screenshot: 'Screenshot',
pdf: 'PDF',
doc: 'Document',
sheet: 'Spreadsheet',
audio: 'Audio',
video: 'Video',
file: 'File',
camera: 'Camera',
screen: 'Screen',
voice: 'Voice',
addImage: 'Add an image',
addFile: 'Add a file',
useCamera: 'Use the camera',
captureScreen: 'Take a screenshot',
recordVoice: 'Record a voice note',
reads: 'What the assistant reads',
upTo: 'up to {v}',
record: 'Start recording',
recording: 'Recording',
paused: 'Paused',
pause: 'Pause',
resume: 'Resume',
stop: 'Stop and keep',
discard: 'Discard',
level: 'Input level',
left: '{v} left',
voiceNote: 'Voice note',
asAudio: 'Send as audio',
asText: 'Insert as text',
transcript: 'Transcript',
uncertain: 'Words in dotted lines were hard to hear.',
micNote: 'The microphone runs only while the red light is on.',
photo: 'Photo',
shutter: 'Take picture',
flip: 'Switch camera',
retake: 'Take again',
attach: 'Attach',
attached: 'Attached',
markArea: 'Drag over the part the assistant should look at',
wholePicture: 'Whole picture',
area: 'Marked area',
pickWindow: 'Choose what to capture',
capture: 'Capture',
cameraNote: 'Pictures stay on this device until you send them.',
viewfinder: 'Camera view',
howTo: 'Take a picture or capture a window|Drag over the part that matters|Attach it and ask your question',
},
de: {
message: 'Nachricht',
placeholder: 'Frag zu deinen Dateien, Fotos oder einer Aufnahme',
send: 'Senden',
sent: 'Gesendet mit {n} Anhängen',
sent1: 'Gesendet mit 1 Anhang',
sentText: 'Gesendet',
attachments: 'Anhänge',
remove: '{name} entfernen',
uploading: 'Wird hochgeladen, {n} %',
ready: 'Bereit',
tooLarge: 'Größer als {mb} MB',
tooLong: 'Länger als {v}',
tooMany: 'Bis zu {n} Anhänge je Nachricht. Der Rest wurde weggelassen.',
unsupported: 'Diese Dateiart kann nicht gelesen werden',
drop: 'Zum Anhängen ablegen',
dropHint: 'Dateien hier ablegen oder auswählen',
choose: 'Dateien wählen',
takePhoto: 'Foto aufnehmen',
limits: '{count} von {max} Anhängen · je bis {mb} MB',
total: '{used} von {max} MB',
image: 'Bild',
screenshot: 'Bildschirmfoto',
pdf: 'PDF',
doc: 'Dokument',
sheet: 'Tabelle',
audio: 'Audio',
video: 'Video',
file: 'Datei',
camera: 'Kamera',
screen: 'Bildschirm',
voice: 'Sprache',
addImage: 'Bild hinzufügen',
addFile: 'Datei hinzufügen',
useCamera: 'Kamera nutzen',
captureScreen: 'Bildschirmfoto aufnehmen',
recordVoice: 'Sprachnachricht aufnehmen',
reads: 'Was der Assistent liest',
upTo: 'bis {v}',
record: 'Aufnahme starten',
recording: 'Nimmt auf',
paused: 'Pausiert',
pause: 'Pausieren',
resume: 'Weiter',
stop: 'Beenden und behalten',
discard: 'Verwerfen',
level: 'Eingangspegel',
left: 'noch {v}',
voiceNote: 'Sprachnachricht',
asAudio: 'Als Audio senden',
asText: 'Als Text einfügen',
transcript: 'Mitschrift',
uncertain: 'Gepunktet unterstrichene Wörter waren schwer zu verstehen.',
micNote: 'Das Mikrofon läuft nur, solange der rote Punkt leuchtet.',
photo: 'Foto',
shutter: 'Foto machen',
flip: 'Kamera wechseln',
retake: 'Neu aufnehmen',
attach: 'Anhängen',
attached: 'Angehängt',
markArea: 'Ziehe über den Teil, den sich der Assistent ansehen soll',
wholePicture: 'Ganzes Bild',
area: 'Markierter Bereich',
pickWindow: 'Wähl, was aufgenommen wird',
capture: 'Aufnehmen',
cameraNote: 'Bilder bleiben auf diesem Gerät, bis du sie sendest.',
viewfinder: 'Kamerabild',
howTo: 'Foto machen oder ein Fenster aufnehmen|Über den wichtigen Teil ziehen|Anhängen und deine Frage stellen',
},
};
type Texts = (typeof TEXTS)['en'];
type P = MultimodalInputProps & { t: Texts };
const fill = (s: string, v: Record<string, string | number>) => s.replace(/\{(\w+)\}/g, (_, k) => String(v[k] ?? ''));
/** State seeded from a prop that starts over when the prop changes (the Studio edits props live) */
function useSynced<T>(value: T) {
const [state, setState] = useState(value);
const key = JSON.stringify(value, (k, v) => (k === 'media' ? undefined : v));
const seen = useRef(key);
if (seen.current !== key) {
seen.current = key;
setState(value);
}
return [state, setState] as const;
}
const CSS =
'@keyframes mmi-up{from{opacity:0;transform:translateY(6px) scale(.97)}}@keyframes mmi-blink{50%{opacity:.25}}@keyframes mmi-flash{from{opacity:.85}to{opacity:0}}' +
'@media (prefers-reduced-motion:no-preference){.mmi-up{animation:mmi-up .35s var(--ease-kit) var(--mmi-d,0ms) both}.mmi-blink{animation:mmi-blink 1.2s ease-in-out infinite}.mmi-flash{animation:mmi-flash .5s ease-out both}}';
function Style() {
return (
<style href="kit-multimodal-input" precedence="default">
{CSS}
</style>
);
}
/* ------------------------------------------------------------------ */
/* Shared parts */
/* ------------------------------------------------------------------ */
const KIND: Record<MediaKind, { icon: IconName; tone: string }> = {
image: { icon: 'image', tone: '--info' },
screenshot: { icon: 'monitor', tone: '--info' },
pdf: { icon: 'file', tone: '--destructive' },
doc: { icon: 'file', tone: '--primary' },
sheet: { icon: 'table', tone: '--success' },
audio: { icon: 'mic', tone: '--warning' },
video: { icon: 'film', tone: '--primary' },
};
const card = 'rounded-2xl border border-border bg-card shadow-[var(--highlight),var(--shadow-md)]';
const btn = 'inline-flex h-11 shrink-0 items-center justify-center gap-1.5 whitespace-nowrap rounded-lg px-3.5 text-[13px] font-medium transition-colors disabled:pointer-events-none disabled:opacity-50 sm:h-9';
const primaryBtn = cn(btn, 'bg-primary text-primary-foreground shadow-[inset_0_1px_0_rgb(255_255_255/0.18),var(--shadow-sm)] hover:bg-[color-mix(in_oklab,var(--primary)_88%,var(--background))]');
const quietBtn = cn(btn, 'border border-border bg-card text-foreground hover:bg-accent');
const toolBtn = 'relative grid size-11 shrink-0 place-items-center rounded-lg text-muted-foreground transition-colors hover:bg-accent hover:text-foreground disabled:opacity-40 sm:size-9';
function useFormat() {
const locale = useLocale();
const intl = locale === 'de' ? 'de-DE' : 'en-US';
const nf1 = new Intl.NumberFormat(intl, { maximumFractionDigits: 1 });
const size = (b: number) => (b >= 1e6 ? `${nf1.format(b / 1e6)} MB` : `${Math.max(1, Math.round(b / 1e3))} KB`);
const mb = (b: number) => nf1.format(b / 1e6);
const time = (s: number) => `${Math.floor(s / 60)}:${String(Math.floor(s % 60)).padStart(2, '0')}`;
const pct = (n: number) => (locale === 'de' ? `${n} %` : `${n}%`);
return { size, mb, time, pct, de: locale === 'de' };
}
function kindOf(file: File): MediaKind | null {
const t = file.type;
if (t.startsWith('image/')) return 'image';
if (t.startsWith('audio/')) return 'audio';
if (t.startsWith('video/')) return 'video';
if (t === 'application/pdf') return 'pdf';
if (/sheet|excel|csv/.test(t) || /\.(xlsx?|csv|numbers)$/i.test(file.name)) return 'sheet';
if (/word|text|rtf|opendocument/.test(t) || /\.(docx?|txt|md|rtf|odt)$/i.test(file.name)) return 'doc';
return null;
}
/** A ring that fills while a file uploads */
function Ring({ value, size = 20 }: { value: number; size?: number }) {
const r = size / 2 - 2;
const c = 2 * Math.PI * r;
return (
<svg aria-hidden width={size} height={size} viewBox={`0 0 ${size} ${size}`} className="-rotate-90">
<circle cx={size / 2} cy={size / 2} r={r} fill="none" stroke="var(--track, color-mix(in oklab, var(--foreground) 12%, transparent))" strokeWidth="2.5" />
<circle cx={size / 2} cy={size / 2} r={r} fill="none" stroke="var(--primary)" strokeWidth="2.5" strokeLinecap="round" strokeDasharray={round(c)} strokeDashoffset={round(c * (1 - value / 100))} className="transition-[stroke-dashoffset] duration-200" />
</svg>
);
}
/** The tile of a file: its picture or an icon in the tone of its kind */
function Thumb({ a, className }: { a: MediaAttachment; className?: string }) {
const k = KIND[a.kind];
return (
<span className={cn('relative grid size-10 shrink-0 place-items-center overflow-hidden rounded-lg', className)} style={a.media ? undefined : { background: `color-mix(in oklab, var(${k.tone}) 15%, var(--card))`, color: `color-mix(in oklab, var(${k.tone}) 75%, var(--foreground))` }}>
{a.media ? (
<span data-clip-ok className="absolute inset-0 [&>*]:size-full [&>img]:object-cover">
{a.media}
</span>
) : (
<Icon name={k.icon} size={17} />
)}
</span>
);
}
/** One attachment as a chip with progress, error and remove */
function Chip({ a, t, onRemove, delay = 0 }: { a: MediaAttachment; t: Texts; onRemove: () => void; delay?: number }) {
const f = useFormat();
const loading = a.progress !== undefined && a.progress < 100 && !a.error;
const meta = a.error ? a.error : loading ? fill(t.uploading, { n: Math.round(a.progress ?? 0) }) : [t[a.kind], a.duration ? f.time(a.duration) : null, f.size(a.size)].filter(Boolean).join(' · ');
return (
<li className="mmi-up flex w-[14rem] max-w-full shrink-0 sm:w-[15rem] items-center gap-2.5 rounded-xl border bg-card py-1.5 pl-1.5 pr-1 shadow-[var(--shadow-sm)]" style={{ '--mmi-d': `${delay}ms`, borderColor: a.error ? 'color-mix(in oklab, var(--destructive) 45%, var(--border))' : 'var(--border)' } as React.CSSProperties}>
<Thumb a={a} className={loading ? 'opacity-60' : undefined} />
<span className="min-w-0 flex-1">
<span className="block truncate text-[12.5px] font-medium text-foreground">{a.name}</span>
<span className={cn('block truncate text-[11.5px] tabular-nums', a.error ? 'text-destructive' : 'text-muted-foreground')} aria-live={loading ? 'off' : 'polite'}>
{meta}
</span>
</span>
{loading && <Ring value={a.progress ?? 0} />}
<button type="button" aria-label={fill(t.remove, { name: a.name })} onClick={onRemove} className="relative grid size-8 shrink-0 place-items-center rounded-md text-muted-foreground transition-colors after:absolute after:-inset-1.5 hover:bg-accent hover:text-foreground">
<Icon name="x" size={14} />
</button>
</li>
);
}
/** Attachments with uploads that move on while in view */
function useAttachments(initial: MediaAttachment[], maxFiles: number, maxSizeMb: number, support: MediaSupport[], t: Texts) {
const [items, setItems] = useSynced(initial);
const [notice, setNotice] = useState('');
const [ref, inView] = useInView<HTMLDivElement>();
const seq = useRef(0);
const uploading = items.some((a) => a.progress !== undefined && a.progress < 100 && !a.error);
useEffect(() => {
if (!inView || !uploading) return;
if (prefersReducedMotion()) return setItems((s) => s.map((a) => (a.error ? a : { ...a, progress: undefined })));
const id = window.setInterval(() => setItems((s) => s.map((a) => (a.progress !== undefined && a.progress < 100 && !a.error ? { ...a, progress: Math.min(100, a.progress + 3 + (a.size % 5)) } : a))), 120);
return () => window.clearInterval(id);
}, [inView, uploading, setItems]);
const limitFor = (k: MediaKind) => support.find((s) => s.kind === k)?.maxMb ?? maxSizeMb;
const add = (list: Omit<MediaAttachment, 'id'>[]) => {
setNotice('');
setItems((s) => {
const room = Math.max(0, maxFiles - s.length);
if (list.length > room) setNotice(fill(t.tooMany, { n: maxFiles }));
const next = list.slice(0, room).map((a) => {
const max = limitFor(a.kind);
return { ...a, id: `new-${++seq.current}`, error: a.error ?? (a.size > max * 1e6 ? fill(t.tooLarge, { mb: max }) : undefined), progress: a.progress ?? 0 };
});
return [...s, ...next];
});
};
const addFiles = (files: FileList | File[]) =>
add(
Array.from(files).map((file) => {
const kind = kindOf(file);
const url = kind === 'image' && typeof URL !== 'undefined' ? URL.createObjectURL(file) : null;
return { name: file.name, kind: kind ?? 'doc', size: file.size, error: kind ? undefined : t.unsupported, media: url ? <img src={url} alt="" /> : undefined };
}),
);
const remove = (id: string) => (setItems((s) => s.filter((a) => a.id !== id)), setNotice(''));
return { items, setItems, add, addFiles, remove, notice, ref };
}
function useSent() {
const [sent, setSent] = useState<string | null>(null);
useEffect(() => {
if (!sent) return;
const id = window.setTimeout(() => setSent(null), 2600);
return () => window.clearTimeout(id);
}, [sent]);
return [sent, setSent] as const;
}
function Stage({ title, description, children, className, wide }: { title?: string; description?: string; children: React.ReactNode; className?: string; wide?: boolean }) {
return (
<section className={cn('kit-glow relative overflow-clip bg-surface py-12 sm:py-16 lg:py-20', className)}>
<Style />
<div className="kit-container">
<div className={cn('mx-auto w-full', wide ? 'max-w-[60rem]' : 'max-w-[46rem]')}>
{(title || description) && (
<Reveal className="mb-6 sm:mb-8">
{title && <h2 className="text-balance text-[22px] font-semibold leading-tight tracking-[-0.02em] text-foreground sm:text-[26px]">{title}</h2>}
{description && <p className="mt-2 max-w-[36rem] text-pretty text-[15px] leading-relaxed text-muted-foreground">{description}</p>}
</Reveal>
)}
{children}
</div>
</div>
</section>
);
}
function Note({ children, icon = 'shield' }: { children: React.ReactNode; icon?: IconName }) {
return (
<p className="mt-3 flex items-start gap-1.5 px-1 text-[12.5px] leading-snug text-muted-foreground">
<Icon name={icon} size={14} className="mt-px shrink-0" />
<span>{children}</span>
</p>
);
}
/** The text field with chips above and a toolbar below */
function Composer({ t, text, setText, items, remove, tools, onSend, sent, busy, limitLine, dropping }: { t: Texts; text: string; setText: (v: string) => void; items: MediaAttachment[]; remove: (id: string) => void; tools?: React.ReactNode; onSend: () => void; sent: string | null; busy?: boolean; limitLine?: React.ReactNode; dropping?: boolean }) {
const uid = useId();
const row = useRef<HTMLUListElement>(null);
const [fade, setFade] = useState('');
// The edge fades only where more chips follow, measured again when chips come or go
useEffect(() => {
const el = row.current;
if (!el) return setFade('');
const update = () => {
const max = el.scrollWidth - el.clientWidth;
setFade(max <= 1 ? '' : el.scrollLeft <= 1 ? 'kit-fade-end' : el.scrollLeft >= max - 1 ? 'kit-fade-start' : 'kit-fade-x');
};
update();
el.addEventListener('scroll', update, { passive: true });
window.addEventListener('resize', update);
return () => (el.removeEventListener('scroll', update), window.removeEventListener('resize', update));
}, [items.length]);
const uploading = items.some((a) => a.progress !== undefined && a.progress < 100 && !a.error);
const ready = items.filter((a) => !a.error);
return (
<form
onSubmit={(e) => {
e.preventDefault();
onSend();
}}
className={cn(card, 'relative p-2 transition-[border-color,box-shadow] duration-200 focus-within:border-[color-mix(in_oklab,var(--primary)_50%,var(--border))] focus-within:shadow-[var(--highlight),var(--shadow-md),0_0_0_3px_color-mix(in_oklab,var(--primary)_16%,transparent)]', dropping && 'border-primary')}
>
{items.length > 0 && (
<ul ref={row} aria-label={t.attachments} className={cn('kit-no-scrollbar relative flex gap-2 overflow-x-auto p-1 pb-2 sm:flex-wrap sm:overflow-visible sm:[mask-image:none]', fade)}>
{items.map((a, i) => (
<Chip key={a.id} a={a} t={t} onRemove={() => remove(a.id)} delay={i * 50} />
))}
</ul>
)}
<label htmlFor={`${uid}-m`} className="sr-only">
{t.message}
</label>
<textarea
id={`${uid}-m`}
rows={2}
value={text}
onChange={(e) => setText(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
onSend();
}
}}
placeholder={t.placeholder}
className="block min-h-11 w-full resize-none bg-transparent px-2.5 py-2 text-[15px] leading-relaxed text-foreground [field-sizing:content] placeholder:text-subtle-foreground focus-visible:outline-none"
/>
<div className="flex items-center gap-1 px-0.5 pt-1">
{tools}
<span className="min-w-0 flex-1 truncate px-1 text-right text-[11.5px] tabular-nums text-subtle-foreground max-sm:hidden">{limitLine}</span>
<button type="submit" disabled={busy || uploading || (!text.trim() && !ready.length)} aria-label={t.send} className={cn(primaryBtn, 'max-sm:ml-auto max-sm:w-11 max-sm:px-0')}>
<Icon name="send" size={15} />
<span className="max-sm:sr-only">{t.send}</span>
</button>
</div>
<p role="status" className="px-2 text-[12.5px] text-success-foreground empty:hidden">
{sent && (
<span className="mmi-up mt-1 inline-flex items-center gap-1.5 pb-1">
<Icon name="check-circle" size={14} />
{sent}
</span>
)}
</p>
</form>
);
}
const sentLine = (t: Texts, n: number) => (n === 0 ? t.sentText : n === 1 ? t.sent1 : fill(t.sent, { n }));
/* ------------------------------------------------------------------ */
/* dropzone */
/* ------------------------------------------------------------------ */
function Dropzone({ t, title, description, defaultValue = '', attachments = [], maxFiles = 8, maxSizeMb = 20, support = [], note, onSubmit, className }: P) {
const f = useFormat();
const uid = useId();
const files = useAttachments(attachments, maxFiles, maxSizeMb, support, t);
const [text, setText] = useState(defaultValue);
const [drag, setDrag] = useState(0);
const [sent, setSent] = useSent();
const input = useRef<HTMLInputElement>(null);
const camera = useRef<HTMLInputElement>(null);
const accept = 'image/*,application/pdf,audio/*,video/*,.doc,.docx,.txt,.md,.csv,.xls,.xlsx';
const send = () => {
const ready = files.items.filter((a) => !a.error);
if (!text.trim() && !ready.length) return;
onSubmit?.({ text, attachments: ready });
setSent(sentLine(t, ready.length));
setText('');
files.setItems([]);
};
const drop = {
onDragEnter: (e: React.DragEvent) => (e.preventDefault(), setDrag((n) => n + 1)),
onDragOver: (e: React.DragEvent) => e.preventDefault(),
onDragLeave: () => setDrag((n) => Math.max(0, n - 1)),
onDrop: (e: React.DragEvent) => {
e.preventDefault();
setDrag(0);
if (e.dataTransfer.files.length) files.addFiles(e.dataTransfer.files);
},
};
const used = files.items.filter((a) => !a.error).reduce((s, a) => s + a.size, 0);
return (
<Stage title={title} description={description} className={className}>
<div ref={files.ref} {...drop} className="relative">
<input ref={input} id={`${uid}-f`} type="file" multiple accept={accept} className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
<input ref={camera} type="file" accept="image/*" capture="environment" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
{/* The zone: a large target from tablets on, two buttons on phones */}
<Reveal delay={40}>
<div className={cn('relative mb-3 hidden overflow-hidden rounded-2xl border-[1.5px] border-dashed p-6 text-center transition-colors duration-200 sm:block', drag ? 'border-primary bg-[color-mix(in_oklab,var(--primary)_8%,var(--card))]' : 'border-border-strong bg-[color-mix(in_oklab,var(--card)_60%,transparent)]')}>
<span aria-hidden className={cn('mx-auto grid size-12 place-items-center rounded-2xl border border-border bg-card text-foreground shadow-[var(--shadow-sm)] transition-transform duration-300 motion-reduce:transition-none', drag && '-translate-y-1 scale-105')}>
<Icon name={drag ? 'download' : 'cloud-upload'} size={22} />
</span>
<p className="mt-3 text-[15px] font-medium text-foreground">{drag ? t.drop : t.dropHint}</p>
<p className="mt-1 text-[12.5px] tabular-nums text-muted-foreground">{fill(t.limits, { count: files.items.length, max: maxFiles, mb: maxSizeMb })}</p>
<button type="button" onClick={() => input.current?.click()} className={cn(quietBtn, 'mt-4')}>
<Icon name="paperclip" size={15} />
{t.choose}
</button>
{support.length > 0 && (
<ul aria-label={t.reads} className="mx-auto mt-5 grid max-w-[38rem] grid-cols-2 gap-2 text-left md:grid-cols-4">
{support.map((s) => (
<li key={s.kind} className="rounded-xl border border-border bg-card p-3">
<p className="flex items-center gap-1.5 text-[12.5px] font-medium text-foreground">
<Icon name={KIND[s.kind].icon} size={14} style={{ color: `color-mix(in oklab, var(${KIND[s.kind].tone}) 75%, var(--foreground))` }} />
{t[s.kind]}
</p>
<p className="mt-1 text-[11.5px] leading-snug text-muted-foreground">{s.reads}</p>
<p className="mt-1.5 text-[11.5px] tabular-nums text-subtle-foreground">{[s.maxMb ? `${s.maxMb} MB` : null, s.maxSeconds ? fill(t.upTo, { v: f.time(s.maxSeconds) }) : null].filter(Boolean).join(' · ')}</p>
</li>
))}
</ul>
)}
</div>
<div className="mb-3 grid grid-cols-2 gap-2 sm:hidden">
<button type="button" onClick={() => camera.current?.click()} className={cn(quietBtn, 'h-12')}>
<Icon name="camera" size={16} />
{t.takePhoto}
</button>
<button type="button" onClick={() => input.current?.click()} className={cn(quietBtn, 'h-12')}>
<Icon name="paperclip" size={16} />
{t.choose}
</button>
</div>
</Reveal>
<Reveal delay={100}>
<Composer
t={t}
text={text}
setText={setText}
items={files.items}
remove={files.remove}
onSend={send}
sent={sent}
dropping={drag > 0}
limitLine={fill(t.total, { used: f.mb(used), max: maxSizeMb * maxFiles })}
tools={
<button type="button" aria-label={t.addFile} title={t.addFile} onClick={() => input.current?.click()} className={toolBtn}>
<Icon name="paperclip" size={18} />
</button>
}
/>
</Reveal>
{files.notice && (
<p role="alert" className="mmi-up mt-2 flex items-start gap-1.5 px-1 text-[12.5px] text-destructive">
<Icon name="alert-circle" size={14} className="mt-px shrink-0" />
{files.notice}
</p>
)}
<p className="mt-2 px-1 text-[12px] tabular-nums text-muted-foreground sm:hidden">{fill(t.limits, { count: files.items.length, max: maxFiles, mb: maxSizeMb })}</p>
{note && <Note>{note}</Note>}
</div>
</Stage>
);
}
/* ------------------------------------------------------------------ */
/* media-toolbar */
/* ------------------------------------------------------------------ */
function MediaToolbar({ t, title, description, defaultValue = '', attachments = [], maxFiles = 8, maxSizeMb = 20, support = [], screen, note, onSubmit, className }: P) {
const f = useFormat();
const files = useAttachments(attachments, maxFiles, maxSizeMb, support, t);
const [text, setText] = useState(defaultValue);
const [sent, setSent] = useSent();
const [rec, setRec] = useState<number | null>(null);
const [flash, setFlash] = useState(0);
const image = useRef<HTMLInputElement>(null);
const file = useRef<HTMLInputElement>(null);
const camera = useRef<HTMLInputElement>(null);
const shots = useRef(0);
const total = maxSizeMb * maxFiles;
const used = files.items.filter((a) => !a.error).reduce((s, a) => s + a.size, 0);
const share = Math.min(100, (used / (total * 1e6)) * 100);
const supported = (k: MediaKind) => !support.length || support.some((s) => s.kind === k);
useEffect(() => {
if (rec === null || prefersReducedMotion()) return;
const id = window.setInterval(() => setRec((r) => (r === null ? r : r + 1)), 1000);
return () => window.clearInterval(id);
}, [rec === null]); // eslint-disable-line react-hooks/exhaustive-deps
const tools: { id: string; icon: IconName; label: string; kind: MediaKind; run: () => void }[] = [
{ id: 'image', icon: 'image', label: t.addImage, kind: 'image', run: () => image.current?.click() },
{ id: 'file', icon: 'paperclip', label: t.addFile, kind: 'pdf', run: () => file.current?.click() },
{ id: 'camera', icon: 'camera', label: t.useCamera, kind: 'image', run: () => camera.current?.click() },
{
id: 'screen',
icon: 'monitor',
label: t.captureScreen,
kind: 'screenshot',
run: () => {
setFlash((n) => n + 1);
files.add([{ name: `${t.screenshot} ${++shots.current}.png`, kind: 'screenshot', size: 840_000 + shots.current * 61_000, media: screen }]);
},
},
{
id: 'voice',
icon: 'mic',
label: t.recordVoice,
kind: 'audio',
run: () => {
if (rec === null) return setRec(0);
files.add([{ name: `${t.voiceNote}.m4a`, kind: 'audio', size: 24_000 * Math.max(1, rec), duration: Math.max(1, rec) }]);
setRec(null);
},
},
];
const send = () => {
const ready = files.items.filter((a) => !a.error);
if (!text.trim() && !ready.length) return;
onSubmit?.({ text, attachments: ready });
setSent(sentLine(t, ready.length));
setText('');
files.setItems([]);
};
return (
<Stage title={title} description={description} className={className}>
<div ref={files.ref}>
<input ref={image} type="file" multiple accept="image/*" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
<input ref={file} type="file" multiple accept="application/pdf,.doc,.docx,.txt,.md,.csv,.xls,.xlsx,audio/*,video/*" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
<input ref={camera} type="file" accept="image/*" capture="environment" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
{flash > 0 && <span key={flash} aria-hidden className="mmi-flash pointer-events-none fixed inset-0 z-50 bg-white opacity-0" />}
<Reveal delay={60}>
<Composer
t={t}
text={text}
setText={setText}
items={files.items}
remove={files.remove}
onSend={send}
sent={sent}
busy={rec !== null}
limitLine={fill(t.limits, { count: files.items.length, max: maxFiles, mb: maxSizeMb })}
tools={
<div role="group" aria-label={t.attachments} className="flex items-center gap-0.5">
{tools.map((x) =>
x.id === 'voice' && rec !== null ? (
<button key={x.id} type="button" onClick={x.run} className={cn(btn, 'gap-2 bg-[color-mix(in_oklab,var(--destructive)_12%,transparent)] px-2.5 text-destructive hover:bg-[color-mix(in_oklab,var(--destructive)_18%,transparent)]')}>
<span aria-hidden className="mmi-blink size-2 rounded-full bg-destructive" />
<span className="tabular-nums">{f.time(rec)}</span>
<span className="sr-only">{t.stop}</span>
<Icon name="stop" size={13} />
</button>
) : (
<button key={x.id} type="button" aria-label={x.label} title={x.label} disabled={!supported(x.kind) || rec !== null} onClick={x.run} className={cn(toolBtn, x.id === 'camera' && 'sm:hidden', x.id === 'screen' && 'max-sm:hidden')}>
<Icon name={x.icon} size={18} />
</button>
),
)}
</div>
}
/>
</Reveal>
{files.notice && (
<p role="alert" className="mmi-up mt-2 flex items-start gap-1.5 px-1 text-[12.5px] text-destructive">
<Icon name="alert-circle" size={14} className="mt-px shrink-0" />
{files.notice}
</p>
)}
{/* What the assistant reads, with the limits per kind */}
<Reveal delay={140} className="mt-4 rounded-2xl border border-border bg-[color-mix(in_oklab,var(--card)_70%,transparent)] p-4">
<div className="flex flex-wrap items-center justify-between gap-x-4 gap-y-2">
<p className="text-[13px] font-medium text-foreground">{t.reads}</p>
<div className="flex min-w-[12rem] items-center gap-2.5 max-sm:w-full">
<span role="meter" aria-label={t.total.replace('{used}', f.mb(used)).replace('{max}', String(total))} aria-valuemin={0} aria-valuemax={100} aria-valuenow={Math.round(share)} className="block h-1.5 flex-1 overflow-hidden rounded-full bg-track">
<span className={cn('block h-full origin-left rounded-full transition-transform duration-500', share > 85 ? 'bg-warning' : 'bg-primary')} style={{ transform: `scaleX(${round(share / 100, 3)})` }} />
</span>
<span className="text-[12px] tabular-nums text-muted-foreground">{fill(t.total, { used: f.mb(used), max: total })}</span>
</div>
</div>
<ul className="mt-3 grid grid-cols-1 gap-x-6 gap-y-2.5 sm:grid-cols-2">
{support.map((s) => (
<li key={s.kind} className="flex items-start gap-2.5">
<span aria-hidden className="grid size-8 shrink-0 place-items-center rounded-lg" style={{ background: `color-mix(in oklab, var(${KIND[s.kind].tone}) 14%, var(--card))`, color: `color-mix(in oklab, var(${KIND[s.kind].tone}) 75%, var(--foreground))` }}>
<Icon name={KIND[s.kind].icon} size={15} />
</span>
<span className="min-w-0">
<span className="flex flex-wrap items-baseline gap-x-2 text-[13px] font-medium text-foreground">
{t[s.kind]}
<span className="text-[11.5px] font-normal tabular-nums text-subtle-foreground">{[s.maxMb ? `${s.maxMb} MB` : null, s.maxSeconds ? fill(t.upTo, { v: f.time(s.maxSeconds) }) : null].filter(Boolean).join(' · ')}</span>
</span>
<span className="block text-[12.5px] leading-snug text-muted-foreground">{s.reads}</span>
</span>
</li>
))}
</ul>
</Reveal>
{note && <Note>{note}</Note>}
</div>
</Stage>
);
}
/* ------------------------------------------------------------------ */
/* voice-note */
/* ------------------------------------------------------------------ */
const BARS = 32;
/** A calm, believable input level: two slow waves and a little noise */
function levelAt(i: number, tick: number) {
const v = 0.35 + 0.3 * Math.sin(tick / 3 + i / 2.4) + 0.22 * Math.sin(tick / 1.7 + i * 1.3) + 0.12 * Math.sin(tick * 1.9 + i * 3.1);
return round(Math.max(0.08, Math.min(1, v)), 2);
}
function VoiceNote({ t, title, description, defaultValue = '', maxSeconds = 300, transcript = '', autoStart, note, onSubmit, className }: P) {
const f = useFormat();
const uid = useId();
const [state, setState] = useState<'idle' | 'recording' | 'paused' | 'done'>('idle');
const [secs, setSecs] = useState(0);
const [tick, setTick] = useState(0);
const [peaks, setPeaks] = useState<number[]>([]);
const [mode, setMode] = useState<'audio' | 'text'>('audio');
const [text, setText] = useState(defaultValue);
const [sent, setSent] = useSent();
const [ref, inView] = useInView<HTMLDivElement>();
const started = useRef(false);
const words = transcript.split(/(\s+)/);
// About two words a second appear while recording, all of them once it stops
const heard = state === 'idle' ? 0 : state === 'done' ? words.length : Math.min(words.length, secs * 4);
useEffect(() => {
if (autoStart && inView && !started.current && !prefersReducedMotion()) {
started.current = true;
setState('recording');
}
}, [autoStart, inView]);
useEffect(() => {
if (state !== 'recording') return;
if (prefersReducedMotion()) {
setSecs((s) => Math.max(s, 14));
return;
}
const a = window.setInterval(() => setTick((n) => n + 1), 110);
const b = window.setInterval(() => setSecs((s) => (s + 1 >= maxSeconds ? (setState('done'), maxSeconds) : s + 1)), 1000);
return () => (window.clearInterval(a), window.clearInterval(b));
}, [state, maxSeconds]);
useEffect(() => {
if (state === 'recording' && tick % 3 === 0) setPeaks((p) => [...p.slice(-47), levelAt(0, tick)]);
}, [tick, state]);
const reset = () => (setState('idle'), setSecs(0), setPeaks([]), setTick(0));
const levels = Array.from({ length: BARS }, (_, i) => (state === 'recording' ? levelAt(i, tick) : state === 'paused' ? 0.1 : levelAt(i, 7) * 0.5));
const share = Math.min(100, (secs / maxSeconds) * 100);
const wave = peaks.length ? peaks : Array.from({ length: 48 }, (_, i) => levelAt(i, 4));
const send = () => {
const att: MediaAttachment[] = state === 'done' && mode === 'audio' ? [{ id: 'voice', name: `${t.voiceNote}.m4a`, kind: 'audio', size: secs * 24_000, duration: secs }] : [];
const body = state === 'done' && mode === 'text' ? [text, transcript.replace(/~/g, '')].filter(Boolean).join(' ') : text;
if (!body.trim() && !att.length) return;
onSubmit?.({ text: body, attachments: att });
setSent(sentLine(t, att.length));
setText('');
reset();
};
return (
<Stage title={title} description={description} className={className}>
<div ref={ref}>
<Reveal effect="scale" className={cn(card, 'overflow-hidden p-0')}>
{/* Recorder */}
<div className={cn('relative px-4 pb-5 pt-6 text-center transition-colors duration-300 sm:px-6', state === 'recording' && 'bg-[color-mix(in_oklab,var(--destructive)_5%,var(--card))]')}>
<p className="flex items-center justify-center gap-2 text-[13px] font-medium" aria-live="polite">
{state === 'recording' && <span aria-hidden className="mmi-blink size-2 rounded-full bg-destructive" />}
<span className={state === 'recording' ? 'text-destructive' : 'text-muted-foreground'}>{state === 'recording' ? t.recording : state === 'paused' ? t.paused : state === 'done' ? t.voiceNote : t.record}</span>
</p>
<p className="mt-1 text-[34px] font-semibold tabular-nums tracking-[-0.03em] text-foreground sm:text-[40px]">{f.time(secs)}</p>
<p className="text-[12px] tabular-nums text-muted-foreground">{fill(t.left, { v: f.time(Math.max(0, maxSeconds - secs)) })}</p>
{state === 'done' ? (
<div className="mmi-up mx-auto mt-5 flex max-w-[28rem] items-center gap-3 rounded-xl border border-border bg-surface p-2.5 text-left">
<span aria-hidden className="grid size-10 shrink-0 place-items-center rounded-lg bg-[color-mix(in_oklab,var(--warning)_15%,var(--card))] text-warning-foreground">
<Icon name="mic" size={17} />
</span>
<svg aria-hidden viewBox={`0 0 ${wave.length * 4} 32`} preserveAspectRatio="none" className="h-8 min-w-0 flex-1">
{wave.map((v, i) => (
<rect key={i} x={i * 4 + 0.75} y={round(16 - v * 14)} width="2.5" height={round(Math.max(2, v * 28))} rx="1.25" fill="color-mix(in oklab, var(--foreground) 55%, transparent)" />
))}
</svg>
<span className="shrink-0 text-[12px] tabular-nums text-muted-foreground">{f.time(secs)}</span>
</div>
) : (
<div role="meter" aria-label={t.level} aria-valuemin={0} aria-valuemax={100} aria-valuenow={Math.round((levels.reduce((s, v) => s + v, 0) / BARS) * 100)} className="mx-auto mt-5 flex h-16 max-w-[28rem] items-center justify-center gap-[3px]">
{levels.map((v, i) => (
<span key={i} aria-hidden className={cn('w-[5px] rounded-full transition-[height] duration-100 ease-linear motion-reduce:transition-none', state === 'recording' ? 'bg-destructive' : 'bg-[color-mix(in_oklab,var(--foreground)_22%,transparent)]')} style={{ height: `${round(8 + v * 56)}px`, opacity: state === 'recording' ? round(0.45 + v * 0.55, 2) : 1 }} />
))}
</div>
)}
<div aria-hidden className="mx-auto mt-4 h-1 max-w-[28rem] overflow-hidden rounded-full bg-track">
<span className="block h-full origin-left rounded-full bg-destructive transition-transform duration-1000 ease-linear" style={{ transform: `scaleX(${round(share / 100, 3)})` }} />
</div>
<div className="mt-5 flex flex-wrap items-center justify-center gap-2">
{state === 'idle' && (
<button type="button" onClick={() => setState('recording')} className="grid size-16 place-items-center rounded-full bg-destructive text-destructive-foreground shadow-[0_6px_20px_color-mix(in_oklab,var(--destructive)_35%,transparent)] transition-transform hover:scale-105 active:scale-95 motion-reduce:transition-none" aria-label={t.record}>
<Icon name="mic" size={24} />
</button>
)}
{(state === 'recording' || state === 'paused') && (
<>
<button type="button" onClick={reset} className={quietBtn}>
<Icon name="trash" size={15} />
{t.discard}
</button>
<button type="button" onClick={() => setState(state === 'recording' ? 'paused' : 'recording')} className={quietBtn}>
<Icon name={state === 'recording' ? 'pause' : 'play'} size={15} />
{state === 'recording' ? t.pause : t.resume}
</button>
<button type="button" onClick={() => setState('done')} className={primaryBtn}>
<Icon name="stop" size={14} />
{t.stop}
</button>
</>
)}
{state === 'done' && (
<>
<button type="button" onClick={reset} className={quietBtn}>
<Icon name="trash" size={15} />
{t.discard}
</button>
<div role="radiogroup" aria-label={t.voiceNote} className="flex gap-1 rounded-lg bg-[color-mix(in_oklab,var(--foreground)_6%,transparent)] p-1">
{(['audio', 'text'] as const).map((m) => (
<button key={m} type="button" role="radio" aria-checked={mode === m} tabIndex={mode === m ? 0 : -1} onClick={() => setMode(m)} onKeyDown={(e) => ['ArrowLeft', 'ArrowRight'].includes(e.key) && (e.preventDefault(), setMode(m === 'audio' ? 'text' : 'audio'), (e.currentTarget.parentElement?.querySelector(`[data-m="${m === 'audio' ? 'text' : 'audio'}"]`) as HTMLElement | null)?.focus())} data-m={m} className={cn('inline-flex h-11 items-center gap-1.5 rounded-md px-3 text-[12.5px] font-medium transition-colors sm:h-8', mode === m ? 'bg-card text-foreground shadow-[var(--highlight),var(--shadow-sm)]' : 'text-muted-foreground hover:text-foreground')}>
<Icon name={m === 'audio' ? 'volume' : 'type'} size={14} />
{m === 'audio' ? t.asAudio : t.asText}
</button>
))}
</div>
</>
)}
</div>
</div>
{/* Transcript as it is heard */}
{transcript && state !== 'idle' && (
<div className="border-t border-border px-4 py-4 sm:px-6">
<p className="text-[12px] font-medium text-muted-foreground">{t.transcript}</p>
<p className="mt-1.5 text-pretty text-[14.5px] leading-relaxed text-foreground" aria-live="polite">
{words.slice(0, heard).map((w, i) =>
w.startsWith('~') ? (
<span key={i} className="underline decoration-[color-mix(in_oklab,var(--warning)_80%,transparent)] decoration-dotted decoration-2 underline-offset-4">
{w.replace(/~/g, '')}
</span>
) : (
<span key={i}>{w.replace(/~/g, '')}</span>
),
)}
{state === 'recording' && <span aria-hidden className="mmi-blink ml-0.5 inline-block h-4 w-0.5 translate-y-0.5 bg-foreground" />}
</p>
{transcript.includes('~') && <p className="mt-2 text-[12px] text-muted-foreground">{t.uncertain}</p>}
</div>
)}
{/* Message */}
<form
onSubmit={(e) => {
e.preventDefault();
send();
}}
className="flex items-end gap-2 border-t border-border p-2"
>
<label htmlFor={`${uid}-m`} className="sr-only">
{t.message}
</label>
<textarea id={`${uid}-m`} rows={1} value={text} onChange={(e) => setText(e.target.value)} placeholder={t.placeholder} className="block min-h-11 w-0 min-w-0 flex-1 resize-none bg-transparent px-2.5 py-2.5 text-[15px] leading-snug text-foreground [field-sizing:content] placeholder:text-subtle-foreground focus-visible:outline-none" />
<button type="submit" disabled={state === 'recording' || state === 'paused' || (!text.trim() && state !== 'done')} aria-label={t.send} className={cn(primaryBtn, 'max-sm:w-11 max-sm:px-0')}>
<Icon name="send" size={15} />
<span className="max-sm:sr-only">{t.send}</span>
</button>
</form>
<p role="status" className="px-4 pb-3 text-[12.5px] text-success-foreground empty:hidden">
{sent && (
<span className="mmi-up inline-flex items-center gap-1.5">
<Icon name="check-circle" size={14} />
{sent}
</span>
)}
</p>
</Reveal>
<Note icon="mic">{note ?? t.micNote}</Note>
</div>
</Stage>
);
}
/* ------------------------------------------------------------------ */
/* camera-capture */
/* ------------------------------------------------------------------ */
type Box = { x: number; y: number; w: number; h: number };
function CameraCapture({ t, title, description, defaultValue = '', preview, screen, windows = [], maxFiles = 8, maxSizeMb = 20, support = [], note, onSubmit, className }: P) {
const uid = useId();
const files = useAttachments([], maxFiles, maxSizeMb, support, t);
const [mode, setMode] = useState<'photo' | 'screen'>('photo');
const [shot, setShot] = useState(false);
const [flash, setFlash] = useState(0);
const [box, setBox] = useState<Box | null>(null);
const [win, setWin] = useState(windows[0]?.id ?? '');
const [text, setText] = useState(defaultValue);
const [sent, setSent] = useSent();
const [front, setFront] = useState(false);
const frame = useRef<HTMLDivElement>(null);
const start = useRef<{ x: number; y: number } | null>(null);
const seq = useRef(0);
const pic = mode === 'photo' ? preview : screen;
const pct = (e: React.PointerEvent) => {
const r = frame.current!.getBoundingClientRect();
return { x: round(Math.max(0, Math.min(100, ((e.clientX - r.left) / r.width) * 100)), 1), y: round(Math.max(0, Math.min(100, ((e.clientY - r.top) / r.height) * 100)), 1) };
};
const capture = () => {
setFlash((n) => n + 1);
setShot(true);
setBox(null);
};
const attach = () => {
const name = mode === 'photo' ? `${t.photo} ${++seq.current}.jpg` : `${windows.find((w) => w.id === win)?.label ?? t.screen}.png`;
files.add([{ name: box ? `${name.replace(/\.\w+$/, '')} · ${t.area}${mode === 'photo' ? '.jpg' : '.png'}` : name, kind: mode === 'photo' ? 'image' : 'screenshot', size: mode === 'photo' ? 2_400_000 : 910_000, media: pic }]);
setShot(false);
setBox(null);
};
const send = () => {
const ready = files.items.filter((a) => !a.error);
if (!text.trim() && !ready.length) return;
onSubmit?.({ text, attachments: ready });
setSent(sentLine(t, ready.length));
setText('');
files.setItems([]);
};
const modes = [
{ id: 'photo' as const, label: t.camera, icon: 'camera' as IconName },
{ id: 'screen' as const, label: t.screen, icon: 'monitor' as IconName },
];
return (
<Stage title={title} description={description} className={className} wide>
<div ref={files.ref} className="grid grid-cols-1 gap-4 lg:grid-cols-[minmax(0,1fr)_22rem] lg:items-start lg:gap-6">
<Reveal effect="scale" className="min-w-0">
<div className="dark rounded-2xl border border-border bg-bezel p-2 shadow-[var(--shadow-lg)]">
<div className="flex items-center justify-between gap-2 px-1 pb-2">
<div role="radiogroup" aria-label={t.capture} className="flex gap-1 rounded-lg bg-[rgb(255_255_255/0.08)] p-1">
{modes.map((m) => (
<button key={m.id} type="button" role="radio" aria-checked={mode === m.id} tabIndex={mode === m.id ? 0 : -1} data-m={m.id} onClick={() => (setMode(m.id), setShot(false), setBox(null))} onKeyDown={(e) => ['ArrowLeft', 'ArrowRight'].includes(e.key) && (e.preventDefault(), setMode(m.id === 'photo' ? 'screen' : 'photo'), setShot(false), setBox(null), (e.currentTarget.parentElement?.querySelector(`[data-m="${m.id === 'photo' ? 'screen' : 'photo'}"]`) as HTMLElement | null)?.focus())} className={cn('inline-flex h-11 items-center gap-1.5 rounded-md px-3 text-[12.5px] font-medium transition-colors sm:h-8', mode === m.id ? 'bg-white text-[#111]' : 'text-white/75 hover:text-white')}>
<Icon name={m.icon} size={14} />
{m.label}
</button>
))}
</div>
{mode === 'photo' && !shot && (
<button type="button" aria-label={t.flip} title={t.flip} onClick={() => setFront((v) => !v)} className="grid size-11 place-items-center rounded-lg text-white/80 hover:bg-white/10 hover:text-white sm:size-9">
<Icon name="refresh" size={17} />
</button>
)}
</div>
{mode === 'screen' && !shot && windows.length > 0 && (
<div
role="radiogroup"
aria-label={t.pickWindow}
// APG radio group: one tab stop, the arrow keys move and choose
onKeyDown={(e) => {
const step = e.key === 'ArrowRight' || e.key === 'ArrowDown' ? 1 : e.key === 'ArrowLeft' || e.key === 'ArrowUp' ? -1 : 0;
if (!step) return;
e.preventDefault();
const i = windows.findIndex((w) => w.id === win);
const next = windows[(i + step + windows.length) % windows.length];
setWin(next.id);
(e.currentTarget.querySelector('[data-w="' + next.id + '"]') as HTMLElement | null)?.focus();
}}
className="kit-no-scrollbar mb-2 flex gap-1.5 overflow-x-auto px-1"
>
{windows.map((w) => (
<button key={w.id} type="button" role="radio" aria-checked={win === w.id} tabIndex={win === w.id ? 0 : -1} data-w={w.id} onClick={() => setWin(w.id)} className={cn('inline-flex h-11 shrink-0 items-center gap-1.5 rounded-lg border px-3 text-[12.5px] font-medium transition-colors sm:h-9', win === w.id ? 'border-white/70 bg-white/12 text-white' : 'border-white/15 text-white/70 hover:text-white')}>
<Icon name={w.icon ?? 'layout'} size={14} />
{w.label}
</button>
))}
</div>
)}
<div
ref={frame}
role={shot ? 'group' : 'img'}
aria-label={shot ? t.markArea : t.viewfinder}
className={cn('relative isolate aspect-[4/3] overflow-hidden rounded-xl bg-black', shot && 'cursor-crosshair touch-none select-none')}
onPointerDown={(e) => {
if (!shot || e.button !== 0) return;
e.currentTarget.setPointerCapture(e.pointerId);
start.current = pct(e);
setBox({ ...start.current, w: 0, h: 0 });
}}
onPointerMove={(e) => {
if (!start.current) return;
const p = pct(e);
const s = start.current;
setBox({ x: Math.min(s.x, p.x), y: Math.min(s.y, p.y), w: round(Math.abs(p.x - s.x), 1), h: round(Math.abs(p.y - s.y), 1) });
}}
onPointerUp={() => {
start.current = null;
setBox((b) => (b && b.w > 3 && b.h > 3 ? b : null));
}}
>
<div data-clip-ok className={cn('absolute inset-0 [&>*]:size-full', mode === 'photo' && front && !shot && '-scale-x-100')}>
{pic}
</div>
{!shot && mode === 'photo' && (
<div aria-hidden className="pointer-events-none absolute inset-6">
{['left-0 top-0 border-l-2 border-t-2 rounded-tl-lg', 'right-0 top-0 border-r-2 border-t-2 rounded-tr-lg', 'bottom-0 left-0 border-b-2 border-l-2 rounded-bl-lg', 'bottom-0 right-0 border-b-2 border-r-2 rounded-br-lg'].map((c) => (
<span key={c} className={cn('absolute size-7 border-white/85', c)} />
))}
</div>
)}
{box && (
<>
<div aria-hidden className="pointer-events-none absolute inset-0 bg-black/45" style={{ clipPath: `polygon(0 0,100% 0,100% 100%,0 100%,0 0,${box.x}% ${box.y}%,${box.x}% ${box.y + box.h}%,${box.x + box.w}% ${box.y + box.h}%,${box.x + box.w}% ${box.y}%,${box.x}% ${box.y}%)`, clipRule: 'evenodd' }} />
<div aria-hidden className="pointer-events-none absolute rounded-md border-2 border-white shadow-[0_0_0_1px_rgb(0_0_0/0.4)]" style={{ left: `${box.x}%`, top: `${box.y}%`, width: `${box.w}%`, height: `${box.h}%` }} />
</>
)}
{shot && !box && (
<span className="pointer-events-none absolute inset-x-0 bottom-3 mx-auto w-fit max-w-[90%] rounded-lg bg-black/65 px-3 py-1.5 text-center text-[12.5px] font-medium text-white backdrop-blur-sm">
<Icon name="crop" size={13} className="mr-1.5 inline align-[-2px]" />
{t.markArea}
</span>
)}
{flash > 0 && <span key={flash} aria-hidden className="mmi-flash pointer-events-none absolute inset-0 bg-white opacity-0" />}
</div>
<div className="flex items-center justify-center gap-3 px-1 pb-1 pt-3">
{shot ? (
<>
<button type="button" onClick={() => (setShot(false), setBox(null))} className="inline-flex h-11 items-center gap-1.5 rounded-lg border border-white/20 px-3.5 text-[13px] font-medium text-white hover:bg-white/10">
<Icon name="rotate-ccw" size={15} />
{t.retake}
</button>
{box && (
<button type="button" onClick={() => setBox(null)} className="inline-flex h-11 items-center gap-1.5 rounded-lg border border-white/20 px-3.5 text-[13px] font-medium text-white hover:bg-white/10">
{t.wholePicture}
</button>
)}
<button type="button" onClick={attach} className="inline-flex h-11 items-center gap-1.5 rounded-lg bg-white px-4 text-[13px] font-semibold text-[#111] hover:bg-white/90">
<Icon name="paperclip" size={15} />
{t.attach}
</button>
</>
) : (
<button type="button" aria-label={mode === 'photo' ? t.shutter : t.capture} onClick={capture} className="grid size-16 place-items-center rounded-full border-4 border-white/85 transition-transform hover:scale-105 active:scale-95 motion-reduce:transition-none">
<span className="size-11 rounded-full bg-white" />
</button>
)}
</div>
</div>
</Reveal>
<Reveal delay={120} className="lg:sticky lg:top-6">
<Composer
t={t}
text={text}
setText={setText}
items={files.items}
remove={files.remove}
onSend={send}
sent={sent}
limitLine={undefined}
tools={
<span className="inline-flex items-center gap-1.5 px-1.5 text-[12px] tabular-nums text-muted-foreground" id={`${uid}-n`}>
<Icon name="paperclip" size={14} />
{files.items.length}/{maxFiles}
</span>
}
/>
<Note>{note ?? t.cameraNote}</Note>
<ol className="mt-5 space-y-3 border-t border-border pt-5 max-lg:hidden">
{t.howTo.split('|').map((x, i) => (
<li key={x} className="flex items-center gap-3 text-[13.5px] text-foreground">
<span aria-hidden className="grid size-7 shrink-0 place-items-center rounded-full border border-border bg-card text-[12px] font-semibold tabular-nums text-muted-foreground">{i + 1}</span>
{x}
</li>
))}
</ol>
</Reveal>
</div>
</Stage>
);
}
/* ------------------------------------------------------------------ */
/**
* Input for an AI that reads more than text: a drop zone with the limits per
* kind, a toolbar for image, file, camera, screenshot and voice with what the
* assistant reads, a voice note with live level, timer and transcript, and a
* camera or screen capture where you mark the part that matters. Chips show
* upload progress, size and errors. Text features (commands, mentions) are
* prompt-input.
*/
export function MultimodalInput({ variant = 'dropzone', labels, ...props }: MultimodalInputProps) {
const t = useTexts(TEXTS, labels);
const p: P = { ...props, t };
switch (variant) {
case 'media-toolbar':
return <MediaToolbar {...p} />;
case 'voice-note':
return <VoiceNote {...p} />;
case 'camera-capture':
return <CameraCapture {...p} />;
default:
return <Dropzone {...p} />;
}
}Frequent questions
How do I install Multimodal input?
With aiw add @ai-website/multimodal-input. The command puts components/ui/multimodal-input.tsx into your project, the code is yours from then on. It brings along icon and reveal. It needs no further npm packages.
Which variants does Multimodal input have?
4 variants, Drop zone, Media toolbar, Voice note and Camera and screen. You pick one with the variant prop, for example variant="media-toolbar".
Can I translate the texts of Multimodal input?
Yes. Multimodal input carries 67 texts of its own in English and German, such as “Message”, “Ask about your files, photos or a recording” and “Send”. The language follows the KitProvider, single texts change through the labels prop.
Can an AI assistant build with Multimodal input?
Yes. Through the MCP server of the aiw CLI, Claude, Cursor, GitHub Copilot, Windsurf, Codex and Gemini find Multimodal input with aiw_components and install it with aiw_add @ai-website/multimodal-input. Name the variant in your prompt, for example variant="dropzone", and the assistant writes your content straight into the props.