Multimodale Eingabe für KI
Eingabe für eine KI, die mehr als Text liest. Eine Ablagezone mit Grenzen je Art, eine Leiste für Bild, Datei, Kamera, Bildschirmfoto und Sprache, eine Sprachnachricht mit Pegel, Zeit und Mitschrift und eine Kamera- oder Bildschirmaufnahme, in der du den wichtigen Teil markierst. Textfunktionen wie Befehle und Erwähnungen stehen in prompt-input.
Dem Assistenten zeigen, nicht nur beschreiben
Viele Fragen lassen sich leichter zeigen als tippen, ein Bildschirmfoto mit Fehler, ein Foto vom Beleg, eine Sprachnachricht unterwegs. Die multimodale Eingabe nimmt all das für einen KI-Assistenten an und sagt vorher, was er liest, niemand lädt also eine Datei umsonst hoch.
Die „Ablagezone“ ist ab Tablet ein großes Ziel für Dateien, mit dem, was jede Art wiegen darf und was gelesen wird, Chips zeigen Fortschritt, Größe und warum eine Datei abgelehnt wird, am Telefon gibt es zwei Knöpfe für Foto und Dateien. Die „Medienleiste“ setzt je Medienart einen Knopf ins Feld, ein Bildschirmfoto landet mit einem Blitz als Chip, der Sprachknopf nimmt direkt in der Leiste auf. Die „Sprachnachricht“ nimmt mit Live-Pegel, Zeit und Restzeit auf und sendet als Audio oder fügt als Text ein, die Mitschrift erscheint beim Sprechen, schwer verständliche Wörter sind markiert. „Kamera und Bildschirm“ öffnet einen Sucher mit Hilfslinien oder ein Fenster zum Aufnehmen, danach ziehst du über den wichtigen Teil und hängst ihn an.
Nenn die Grenzen vor dem Upload, nicht in einer Fehlermeldung danach. Lass Menschen ein Bildschirmfoto auf den wichtigen Teil zuschneiden, die Antwort wird besser und die Anfrage kleiner. Textfunktionen wie Befehle und Erwähnungen stecken in der Prompt-Eingabe, beide arbeiten nebeneinander.
Variants
Details
Was Multimodale Eingabe mitbringt, aus Registry und Quellcode gelesen.
- Kategorie
- AI und Chat
- Bereich
- App
- Variants
- 4
- Version
- 1.0.0 · seit 1.2.0
- Datei
- components/ui/multimodal-input.tsx
- Bringt mit
- utilskiticonreveal
- npm-Pakete
- Keine
- Sprachen
- Englisch und Deutsch, 67 eigene Texte
- Studio
- Im Code anpassen
- In Sammlungen
- KI-Produkt
- Mit KI bauen
- Per MCP in Claude, Cursor, Copilot und Codex
Props
Die Schnittstelle MultimodalInputProps aus components/ui/multimodal-input.tsx. Beschreibungen stammen aus den Kommentaren im Code und sind auf Englisch.
variantoptionalMultimodalInputVariantdropzonemedia-toolbarvoice-notecamera-capture
titleoptionalstringdescriptionoptionalstringplaceholderoptionalstringdefaultValueoptionalstringText already in the field
attachmentsoptionalMediaAttachment[]maxFilesoptionalnumberMost attachments per message
maxSizeMboptionalnumberLargest file in MB, unless a kind sets its own
supportoptionalMediaSupport[]What the model reads per kind
maxSecondsoptionalnumbervoice-note: longest recording in seconds
transcriptoptionalstringvoice-note: the text heard, shown as the transcript
autoStartoptionalbooleanvoice-note: start recording once in view
previewoptionalReact.ReactNodecamera-capture: the picture the camera sees
screenoptionalReact.ReactNodecamera-capture: the picture a screenshot shows
windowsoptionalCaptureWindow[]camera-capture: windows to choose in screenshot mode
noteoptionalstringLine under the field, e.g. what happens with files and audio
onSubmitoptional(value: MultimodalSubmit) => void
Eingebaute Texte
Diese Texte schreibt Multimodale Eingabe selbst. Die Sprache folgt dem KitProvider, über die Prop labels ersetzt du einzelne.
Alle 67 Texte anzeigen
| Schlüssel | Englisch | Deutsch |
|---|---|---|
| message | Englisch Message | Deutsch Nachricht |
| placeholder | Englisch Ask about your files, photos or a recording | Deutsch Frag zu deinen Dateien, Fotos oder einer Aufnahme |
| send | Englisch Send | Deutsch Senden |
| sent | Englisch Sent with {n} attachments | Deutsch Gesendet mit {n} Anhängen |
| sent1 | Englisch Sent with 1 attachment | Deutsch Gesendet mit 1 Anhang |
| sentText | Englisch Sent | Deutsch Gesendet |
| attachments | Englisch Attachments | Deutsch Anhänge |
| remove | Englisch Remove {name} | Deutsch {name} entfernen |
| uploading | Englisch Uploading, {n}% | Deutsch Wird hochgeladen, {n} % |
| ready | Englisch Ready | Deutsch Bereit |
| tooLarge | Englisch Larger than {mb} MB | Deutsch Größer als {mb} MB |
| tooLong | Englisch Longer than {v} | Deutsch Länger als {v} |
| tooMany | Englisch Up to {n} attachments per message. The rest was left out. | Deutsch Bis zu {n} Anhänge je Nachricht. Der Rest wurde weggelassen. |
| unsupported | Englisch This file type cannot be read | Deutsch Diese Dateiart kann nicht gelesen werden |
| drop | Englisch Drop to attach | Deutsch Zum Anhängen ablegen |
| dropHint | Englisch Drop files here or choose them | Deutsch Dateien hier ablegen oder auswählen |
| choose | Englisch Choose files | Deutsch Dateien wählen |
| takePhoto | Englisch Take a photo | Deutsch Foto aufnehmen |
| limits | Englisch {count} of {max} attachments · up to {mb} MB each | Deutsch {count} von {max} Anhängen · je bis {mb} MB |
| total | Englisch {used} of {max} MB | Deutsch {used} von {max} MB |
| image | Englisch Image | Deutsch Bild |
| screenshot | Englisch Screenshot | Deutsch Bildschirmfoto |
| Englisch PDF | Deutsch PDF | |
| doc | Englisch Document | Deutsch Dokument |
| sheet | Englisch Spreadsheet | Deutsch Tabelle |
| audio | Englisch Audio | Deutsch Audio |
| video | Englisch Video | Deutsch Video |
| file | Englisch File | Deutsch Datei |
| camera | Englisch Camera | Deutsch Kamera |
| screen | Englisch Screen | Deutsch Bildschirm |
| voice | Englisch Voice | Deutsch Sprache |
| addImage | Englisch Add an image | Deutsch Bild hinzufügen |
| addFile | Englisch Add a file | Deutsch Datei hinzufügen |
| useCamera | Englisch Use the camera | Deutsch Kamera nutzen |
| captureScreen | Englisch Take a screenshot | Deutsch Bildschirmfoto aufnehmen |
| recordVoice | Englisch Record a voice note | Deutsch Sprachnachricht aufnehmen |
| reads | Englisch What the assistant reads | Deutsch Was der Assistent liest |
| upTo | Englisch up to {v} | Deutsch bis {v} |
| record | Englisch Start recording | Deutsch Aufnahme starten |
| recording | Englisch Recording | Deutsch Nimmt auf |
| paused | Englisch Paused | Deutsch Pausiert |
| pause | Englisch Pause | Deutsch Pausieren |
| resume | Englisch Resume | Deutsch Weiter |
| stop | Englisch Stop and keep | Deutsch Beenden und behalten |
| discard | Englisch Discard | Deutsch Verwerfen |
| level | Englisch Input level | Deutsch Eingangspegel |
| left | Englisch {v} left | Deutsch noch {v} |
| voiceNote | Englisch Voice note | Deutsch Sprachnachricht |
| asAudio | Englisch Send as audio | Deutsch Als Audio senden |
| asText | Englisch Insert as text | Deutsch Als Text einfügen |
| transcript | Englisch Transcript | Deutsch Mitschrift |
| uncertain | Englisch Words in dotted lines were hard to hear. | Deutsch Gepunktet unterstrichene Wörter waren schwer zu verstehen. |
| micNote | Englisch The microphone runs only while the red light is on. | Deutsch Das Mikrofon läuft nur, solange der rote Punkt leuchtet. |
| photo | Englisch Photo | Deutsch Foto |
| shutter | Englisch Take picture | Deutsch Foto machen |
| flip | Englisch Switch camera | Deutsch Kamera wechseln |
| retake | Englisch Take again | Deutsch Neu aufnehmen |
| attach | Englisch Attach | Deutsch Anhängen |
| attached | Englisch Attached | Deutsch Angehängt |
| markArea | Englisch Drag over the part the assistant should look at | Deutsch Ziehe über den Teil, den sich der Assistent ansehen soll |
| wholePicture | Englisch Whole picture | Deutsch Ganzes Bild |
| area | Englisch Marked area | Deutsch Markierter Bereich |
| pickWindow | Englisch Choose what to capture | Deutsch Wähl, was aufgenommen wird |
| capture | Englisch Capture | Deutsch Aufnehmen |
| cameraNote | Englisch Pictures stay on this device until you send them. | Deutsch Bilder bleiben auf diesem Gerät, bis du sie sendest. |
| viewfinder | Englisch Camera view | Deutsch Kamerabild |
| howTo | Englisch Take a picture or capture a window|Drag over the part that matters|Attach it and ask your question | Deutsch Foto machen oder ein Fenster aufnehmen|Über den wichtigen Teil ziehen|Anhängen und deine Frage stellen |
Barrierefreiheit
Aus dem Quellcode der Component abgeleitet.
- Multimodale Eingabe reagiert selbst auf die Tasten Enter, Pfeil hoch, Pfeil runter, Pfeil links, Pfeil rechts und Leertaste.
- Setzt aria-checked, aria-label, aria-live, aria-valuemax, aria-valuemin und aria-valuenow, damit Screenreader Zweck und Zustand ansagen.
- Nutzt die Rollen alert, group, meter, radio, radiogroup, shot und status.
- Meldet Änderungen über eine Live-Region, ohne den Fokus zu verschieben.
- Jedes Feld hat ein Label, sichtbar oder für Screenreader.
- Ergänzt Text für Screenreader, wo Icons und Grafiken allein stehen.
- Zeigt bei Bedienung mit der Tastatur einen sichtbaren Fokusring.
- Der Server rendert den Startzustand vollständig, JavaScript ergänzt nur die Interaktion.
Bewegung
- Erscheint beim ersten Sichtkontakt mit einem kurzen Auftritt (kit-rise, useInView).
- Bewegt sich mit dem Scrollen, der Fortschritt folgt der Position auf der Seite.
- Bringt eigene Animationen mit, ruhig getaktet und ohne Wackeln.
- Bei reduzierter Bewegung (prefers-reduced-motion) bleibt sie ruhig. 4 Stellen im Code schalten Bewegung dann ab.
Quellcode
Die Datei, die aiw add in dein Projekt legt. Sie gehört danach dir.
Datei anzeigen, 1.160 Zeilen
'use client';
import { useEffect, useId, useRef, useState } from 'react';
import { cn, prefersReducedMotion, round, useInView, useLocale, useTexts } from '@/lib/kit';
import { Icon, type IconName } from '@/components/ui/icon';
import { Reveal } from '@/components/ui/reveal';
export type MultimodalInputVariant = 'dropzone' | 'media-toolbar' | 'voice-note' | 'camera-capture';
export type MediaKind = 'image' | 'screenshot' | 'pdf' | 'doc' | 'sheet' | 'audio' | 'video';
export interface MediaAttachment {
id: string;
name: string;
kind: MediaKind;
/** Size in bytes */
size: number;
/** 0 to 100, below 100 the file is still uploading */
progress?: number;
/** A message when the file cannot be used */
error?: string;
/** Audio and video length in seconds */
duration?: number;
/** Preview for images and screenshots, a drawn motif or your own <img> */
media?: React.ReactNode;
}
/** What the model reads per kind of media, shown as plain words */
export interface MediaSupport {
kind: MediaKind;
/** e.g. "Reads text and tables" */
reads: string;
/** Largest file in MB */
maxMb?: number;
/** Longest audio or video in seconds */
maxSeconds?: number;
}
/** camera-capture: a window to share in screenshot mode */
export interface CaptureWindow {
id: string;
label: string;
icon?: IconName;
}
export interface MultimodalSubmit {
text: string;
attachments: MediaAttachment[];
}
export interface MultimodalInputProps {
variant?: MultimodalInputVariant;
title?: string;
description?: string;
placeholder?: string;
/** Text already in the field */
defaultValue?: string;
attachments?: MediaAttachment[];
/** Most attachments per message */
maxFiles?: number;
/** Largest file in MB, unless a kind sets its own */
maxSizeMb?: number;
/** What the model reads per kind */
support?: MediaSupport[];
/** voice-note: longest recording in seconds */
maxSeconds?: number;
/** voice-note: the text heard, shown as the transcript */
transcript?: string;
/** voice-note: start recording once in view */
autoStart?: boolean;
/** camera-capture: the picture the camera sees */
preview?: React.ReactNode;
/** camera-capture: the picture a screenshot shows */
screen?: React.ReactNode;
/** camera-capture: windows to choose in screenshot mode */
windows?: CaptureWindow[];
/** Line under the field, e.g. what happens with files and audio */
note?: string;
onSubmit?: (value: MultimodalSubmit) => void;
labels?: Partial<(typeof TEXTS)['en']>;
className?: string;
}
const TEXTS = {
en: {
message: 'Message',
placeholder: 'Ask about your files, photos or a recording',
send: 'Send',
sent: 'Sent with {n} attachments',
sent1: 'Sent with 1 attachment',
sentText: 'Sent',
attachments: 'Attachments',
remove: 'Remove {name}',
uploading: 'Uploading, {n}%',
ready: 'Ready',
tooLarge: 'Larger than {mb} MB',
tooLong: 'Longer than {v}',
tooMany: 'Up to {n} attachments per message. The rest was left out.',
unsupported: 'This file type cannot be read',
drop: 'Drop to attach',
dropHint: 'Drop files here or choose them',
choose: 'Choose files',
takePhoto: 'Take a photo',
limits: '{count} of {max} attachments · up to {mb} MB each',
total: '{used} of {max} MB',
image: 'Image',
screenshot: 'Screenshot',
pdf: 'PDF',
doc: 'Document',
sheet: 'Spreadsheet',
audio: 'Audio',
video: 'Video',
file: 'File',
camera: 'Camera',
screen: 'Screen',
voice: 'Voice',
addImage: 'Add an image',
addFile: 'Add a file',
useCamera: 'Use the camera',
captureScreen: 'Take a screenshot',
recordVoice: 'Record a voice note',
reads: 'What the assistant reads',
upTo: 'up to {v}',
record: 'Start recording',
recording: 'Recording',
paused: 'Paused',
pause: 'Pause',
resume: 'Resume',
stop: 'Stop and keep',
discard: 'Discard',
level: 'Input level',
left: '{v} left',
voiceNote: 'Voice note',
asAudio: 'Send as audio',
asText: 'Insert as text',
transcript: 'Transcript',
uncertain: 'Words in dotted lines were hard to hear.',
micNote: 'The microphone runs only while the red light is on.',
photo: 'Photo',
shutter: 'Take picture',
flip: 'Switch camera',
retake: 'Take again',
attach: 'Attach',
attached: 'Attached',
markArea: 'Drag over the part the assistant should look at',
wholePicture: 'Whole picture',
area: 'Marked area',
pickWindow: 'Choose what to capture',
capture: 'Capture',
cameraNote: 'Pictures stay on this device until you send them.',
viewfinder: 'Camera view',
howTo: 'Take a picture or capture a window|Drag over the part that matters|Attach it and ask your question',
},
de: {
message: 'Nachricht',
placeholder: 'Frag zu deinen Dateien, Fotos oder einer Aufnahme',
send: 'Senden',
sent: 'Gesendet mit {n} Anhängen',
sent1: 'Gesendet mit 1 Anhang',
sentText: 'Gesendet',
attachments: 'Anhänge',
remove: '{name} entfernen',
uploading: 'Wird hochgeladen, {n} %',
ready: 'Bereit',
tooLarge: 'Größer als {mb} MB',
tooLong: 'Länger als {v}',
tooMany: 'Bis zu {n} Anhänge je Nachricht. Der Rest wurde weggelassen.',
unsupported: 'Diese Dateiart kann nicht gelesen werden',
drop: 'Zum Anhängen ablegen',
dropHint: 'Dateien hier ablegen oder auswählen',
choose: 'Dateien wählen',
takePhoto: 'Foto aufnehmen',
limits: '{count} von {max} Anhängen · je bis {mb} MB',
total: '{used} von {max} MB',
image: 'Bild',
screenshot: 'Bildschirmfoto',
pdf: 'PDF',
doc: 'Dokument',
sheet: 'Tabelle',
audio: 'Audio',
video: 'Video',
file: 'Datei',
camera: 'Kamera',
screen: 'Bildschirm',
voice: 'Sprache',
addImage: 'Bild hinzufügen',
addFile: 'Datei hinzufügen',
useCamera: 'Kamera nutzen',
captureScreen: 'Bildschirmfoto aufnehmen',
recordVoice: 'Sprachnachricht aufnehmen',
reads: 'Was der Assistent liest',
upTo: 'bis {v}',
record: 'Aufnahme starten',
recording: 'Nimmt auf',
paused: 'Pausiert',
pause: 'Pausieren',
resume: 'Weiter',
stop: 'Beenden und behalten',
discard: 'Verwerfen',
level: 'Eingangspegel',
left: 'noch {v}',
voiceNote: 'Sprachnachricht',
asAudio: 'Als Audio senden',
asText: 'Als Text einfügen',
transcript: 'Mitschrift',
uncertain: 'Gepunktet unterstrichene Wörter waren schwer zu verstehen.',
micNote: 'Das Mikrofon läuft nur, solange der rote Punkt leuchtet.',
photo: 'Foto',
shutter: 'Foto machen',
flip: 'Kamera wechseln',
retake: 'Neu aufnehmen',
attach: 'Anhängen',
attached: 'Angehängt',
markArea: 'Ziehe über den Teil, den sich der Assistent ansehen soll',
wholePicture: 'Ganzes Bild',
area: 'Markierter Bereich',
pickWindow: 'Wähl, was aufgenommen wird',
capture: 'Aufnehmen',
cameraNote: 'Bilder bleiben auf diesem Gerät, bis du sie sendest.',
viewfinder: 'Kamerabild',
howTo: 'Foto machen oder ein Fenster aufnehmen|Über den wichtigen Teil ziehen|Anhängen und deine Frage stellen',
},
};
type Texts = (typeof TEXTS)['en'];
type P = MultimodalInputProps & { t: Texts };
const fill = (s: string, v: Record<string, string | number>) => s.replace(/\{(\w+)\}/g, (_, k) => String(v[k] ?? ''));
/** State seeded from a prop that starts over when the prop changes (the Studio edits props live) */
function useSynced<T>(value: T) {
const [state, setState] = useState(value);
const key = JSON.stringify(value, (k, v) => (k === 'media' ? undefined : v));
const seen = useRef(key);
if (seen.current !== key) {
seen.current = key;
setState(value);
}
return [state, setState] as const;
}
const CSS =
'@keyframes mmi-up{from{opacity:0;transform:translateY(6px) scale(.97)}}@keyframes mmi-blink{50%{opacity:.25}}@keyframes mmi-flash{from{opacity:.85}to{opacity:0}}' +
'@media (prefers-reduced-motion:no-preference){.mmi-up{animation:mmi-up .35s var(--ease-kit) var(--mmi-d,0ms) both}.mmi-blink{animation:mmi-blink 1.2s ease-in-out infinite}.mmi-flash{animation:mmi-flash .5s ease-out both}}';
function Style() {
return (
<style href="kit-multimodal-input" precedence="default">
{CSS}
</style>
);
}
/* ------------------------------------------------------------------ */
/* Shared parts */
/* ------------------------------------------------------------------ */
const KIND: Record<MediaKind, { icon: IconName; tone: string }> = {
image: { icon: 'image', tone: '--info' },
screenshot: { icon: 'monitor', tone: '--info' },
pdf: { icon: 'file', tone: '--destructive' },
doc: { icon: 'file', tone: '--primary' },
sheet: { icon: 'table', tone: '--success' },
audio: { icon: 'mic', tone: '--warning' },
video: { icon: 'film', tone: '--primary' },
};
const card = 'rounded-2xl border border-border bg-card shadow-[var(--highlight),var(--shadow-md)]';
const btn = 'inline-flex h-11 shrink-0 items-center justify-center gap-1.5 whitespace-nowrap rounded-lg px-3.5 text-[13px] font-medium transition-colors disabled:pointer-events-none disabled:opacity-50 sm:h-9';
const primaryBtn = cn(btn, 'bg-primary text-primary-foreground shadow-[inset_0_1px_0_rgb(255_255_255/0.18),var(--shadow-sm)] hover:bg-[color-mix(in_oklab,var(--primary)_88%,var(--background))]');
const quietBtn = cn(btn, 'border border-border bg-card text-foreground hover:bg-accent');
const toolBtn = 'relative grid size-11 shrink-0 place-items-center rounded-lg text-muted-foreground transition-colors hover:bg-accent hover:text-foreground disabled:opacity-40 sm:size-9';
function useFormat() {
const locale = useLocale();
const intl = locale === 'de' ? 'de-DE' : 'en-US';
const nf1 = new Intl.NumberFormat(intl, { maximumFractionDigits: 1 });
const size = (b: number) => (b >= 1e6 ? `${nf1.format(b / 1e6)} MB` : `${Math.max(1, Math.round(b / 1e3))} KB`);
const mb = (b: number) => nf1.format(b / 1e6);
const time = (s: number) => `${Math.floor(s / 60)}:${String(Math.floor(s % 60)).padStart(2, '0')}`;
const pct = (n: number) => (locale === 'de' ? `${n} %` : `${n}%`);
return { size, mb, time, pct, de: locale === 'de' };
}
function kindOf(file: File): MediaKind | null {
const t = file.type;
if (t.startsWith('image/')) return 'image';
if (t.startsWith('audio/')) return 'audio';
if (t.startsWith('video/')) return 'video';
if (t === 'application/pdf') return 'pdf';
if (/sheet|excel|csv/.test(t) || /\.(xlsx?|csv|numbers)$/i.test(file.name)) return 'sheet';
if (/word|text|rtf|opendocument/.test(t) || /\.(docx?|txt|md|rtf|odt)$/i.test(file.name)) return 'doc';
return null;
}
/** A ring that fills while a file uploads */
function Ring({ value, size = 20 }: { value: number; size?: number }) {
const r = size / 2 - 2;
const c = 2 * Math.PI * r;
return (
<svg aria-hidden width={size} height={size} viewBox={`0 0 ${size} ${size}`} className="-rotate-90">
<circle cx={size / 2} cy={size / 2} r={r} fill="none" stroke="var(--track, color-mix(in oklab, var(--foreground) 12%, transparent))" strokeWidth="2.5" />
<circle cx={size / 2} cy={size / 2} r={r} fill="none" stroke="var(--primary)" strokeWidth="2.5" strokeLinecap="round" strokeDasharray={round(c)} strokeDashoffset={round(c * (1 - value / 100))} className="transition-[stroke-dashoffset] duration-200" />
</svg>
);
}
/** The tile of a file: its picture or an icon in the tone of its kind */
function Thumb({ a, className }: { a: MediaAttachment; className?: string }) {
const k = KIND[a.kind];
return (
<span className={cn('relative grid size-10 shrink-0 place-items-center overflow-hidden rounded-lg', className)} style={a.media ? undefined : { background: `color-mix(in oklab, var(${k.tone}) 15%, var(--card))`, color: `color-mix(in oklab, var(${k.tone}) 75%, var(--foreground))` }}>
{a.media ? (
<span data-clip-ok className="absolute inset-0 [&>*]:size-full [&>img]:object-cover">
{a.media}
</span>
) : (
<Icon name={k.icon} size={17} />
)}
</span>
);
}
/** One attachment as a chip with progress, error and remove */
function Chip({ a, t, onRemove, delay = 0 }: { a: MediaAttachment; t: Texts; onRemove: () => void; delay?: number }) {
const f = useFormat();
const loading = a.progress !== undefined && a.progress < 100 && !a.error;
const meta = a.error ? a.error : loading ? fill(t.uploading, { n: Math.round(a.progress ?? 0) }) : [t[a.kind], a.duration ? f.time(a.duration) : null, f.size(a.size)].filter(Boolean).join(' · ');
return (
<li className="mmi-up flex w-[14rem] max-w-full shrink-0 sm:w-[15rem] items-center gap-2.5 rounded-xl border bg-card py-1.5 pl-1.5 pr-1 shadow-[var(--shadow-sm)]" style={{ '--mmi-d': `${delay}ms`, borderColor: a.error ? 'color-mix(in oklab, var(--destructive) 45%, var(--border))' : 'var(--border)' } as React.CSSProperties}>
<Thumb a={a} className={loading ? 'opacity-60' : undefined} />
<span className="min-w-0 flex-1">
<span className="block truncate text-[12.5px] font-medium text-foreground">{a.name}</span>
<span className={cn('block truncate text-[11.5px] tabular-nums', a.error ? 'text-destructive' : 'text-muted-foreground')} aria-live={loading ? 'off' : 'polite'}>
{meta}
</span>
</span>
{loading && <Ring value={a.progress ?? 0} />}
<button type="button" aria-label={fill(t.remove, { name: a.name })} onClick={onRemove} className="relative grid size-8 shrink-0 place-items-center rounded-md text-muted-foreground transition-colors after:absolute after:-inset-1.5 hover:bg-accent hover:text-foreground">
<Icon name="x" size={14} />
</button>
</li>
);
}
/** Attachments with uploads that move on while in view */
function useAttachments(initial: MediaAttachment[], maxFiles: number, maxSizeMb: number, support: MediaSupport[], t: Texts) {
const [items, setItems] = useSynced(initial);
const [notice, setNotice] = useState('');
const [ref, inView] = useInView<HTMLDivElement>();
const seq = useRef(0);
const uploading = items.some((a) => a.progress !== undefined && a.progress < 100 && !a.error);
useEffect(() => {
if (!inView || !uploading) return;
if (prefersReducedMotion()) return setItems((s) => s.map((a) => (a.error ? a : { ...a, progress: undefined })));
const id = window.setInterval(() => setItems((s) => s.map((a) => (a.progress !== undefined && a.progress < 100 && !a.error ? { ...a, progress: Math.min(100, a.progress + 3 + (a.size % 5)) } : a))), 120);
return () => window.clearInterval(id);
}, [inView, uploading, setItems]);
const limitFor = (k: MediaKind) => support.find((s) => s.kind === k)?.maxMb ?? maxSizeMb;
const add = (list: Omit<MediaAttachment, 'id'>[]) => {
setNotice('');
setItems((s) => {
const room = Math.max(0, maxFiles - s.length);
if (list.length > room) setNotice(fill(t.tooMany, { n: maxFiles }));
const next = list.slice(0, room).map((a) => {
const max = limitFor(a.kind);
return { ...a, id: `new-${++seq.current}`, error: a.error ?? (a.size > max * 1e6 ? fill(t.tooLarge, { mb: max }) : undefined), progress: a.progress ?? 0 };
});
return [...s, ...next];
});
};
const addFiles = (files: FileList | File[]) =>
add(
Array.from(files).map((file) => {
const kind = kindOf(file);
const url = kind === 'image' && typeof URL !== 'undefined' ? URL.createObjectURL(file) : null;
return { name: file.name, kind: kind ?? 'doc', size: file.size, error: kind ? undefined : t.unsupported, media: url ? <img src={url} alt="" /> : undefined };
}),
);
const remove = (id: string) => (setItems((s) => s.filter((a) => a.id !== id)), setNotice(''));
return { items, setItems, add, addFiles, remove, notice, ref };
}
function useSent() {
const [sent, setSent] = useState<string | null>(null);
useEffect(() => {
if (!sent) return;
const id = window.setTimeout(() => setSent(null), 2600);
return () => window.clearTimeout(id);
}, [sent]);
return [sent, setSent] as const;
}
function Stage({ title, description, children, className, wide }: { title?: string; description?: string; children: React.ReactNode; className?: string; wide?: boolean }) {
return (
<section className={cn('kit-glow relative overflow-clip bg-surface py-12 sm:py-16 lg:py-20', className)}>
<Style />
<div className="kit-container">
<div className={cn('mx-auto w-full', wide ? 'max-w-[60rem]' : 'max-w-[46rem]')}>
{(title || description) && (
<Reveal className="mb-6 sm:mb-8">
{title && <h2 className="text-balance text-[22px] font-semibold leading-tight tracking-[-0.02em] text-foreground sm:text-[26px]">{title}</h2>}
{description && <p className="mt-2 max-w-[36rem] text-pretty text-[15px] leading-relaxed text-muted-foreground">{description}</p>}
</Reveal>
)}
{children}
</div>
</div>
</section>
);
}
function Note({ children, icon = 'shield' }: { children: React.ReactNode; icon?: IconName }) {
return (
<p className="mt-3 flex items-start gap-1.5 px-1 text-[12.5px] leading-snug text-muted-foreground">
<Icon name={icon} size={14} className="mt-px shrink-0" />
<span>{children}</span>
</p>
);
}
/** The text field with chips above and a toolbar below */
function Composer({ t, text, setText, items, remove, tools, onSend, sent, busy, limitLine, dropping }: { t: Texts; text: string; setText: (v: string) => void; items: MediaAttachment[]; remove: (id: string) => void; tools?: React.ReactNode; onSend: () => void; sent: string | null; busy?: boolean; limitLine?: React.ReactNode; dropping?: boolean }) {
const uid = useId();
const row = useRef<HTMLUListElement>(null);
const [fade, setFade] = useState('');
// The edge fades only where more chips follow, measured again when chips come or go
useEffect(() => {
const el = row.current;
if (!el) return setFade('');
const update = () => {
const max = el.scrollWidth - el.clientWidth;
setFade(max <= 1 ? '' : el.scrollLeft <= 1 ? 'kit-fade-end' : el.scrollLeft >= max - 1 ? 'kit-fade-start' : 'kit-fade-x');
};
update();
el.addEventListener('scroll', update, { passive: true });
window.addEventListener('resize', update);
return () => (el.removeEventListener('scroll', update), window.removeEventListener('resize', update));
}, [items.length]);
const uploading = items.some((a) => a.progress !== undefined && a.progress < 100 && !a.error);
const ready = items.filter((a) => !a.error);
return (
<form
onSubmit={(e) => {
e.preventDefault();
onSend();
}}
className={cn(card, 'relative p-2 transition-[border-color,box-shadow] duration-200 focus-within:border-[color-mix(in_oklab,var(--primary)_50%,var(--border))] focus-within:shadow-[var(--highlight),var(--shadow-md),0_0_0_3px_color-mix(in_oklab,var(--primary)_16%,transparent)]', dropping && 'border-primary')}
>
{items.length > 0 && (
<ul ref={row} aria-label={t.attachments} className={cn('kit-no-scrollbar relative flex gap-2 overflow-x-auto p-1 pb-2 sm:flex-wrap sm:overflow-visible sm:[mask-image:none]', fade)}>
{items.map((a, i) => (
<Chip key={a.id} a={a} t={t} onRemove={() => remove(a.id)} delay={i * 50} />
))}
</ul>
)}
<label htmlFor={`${uid}-m`} className="sr-only">
{t.message}
</label>
<textarea
id={`${uid}-m`}
rows={2}
value={text}
onChange={(e) => setText(e.target.value)}
onKeyDown={(e) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
onSend();
}
}}
placeholder={t.placeholder}
className="block min-h-11 w-full resize-none bg-transparent px-2.5 py-2 text-[15px] leading-relaxed text-foreground [field-sizing:content] placeholder:text-subtle-foreground focus-visible:outline-none"
/>
<div className="flex items-center gap-1 px-0.5 pt-1">
{tools}
<span className="min-w-0 flex-1 truncate px-1 text-right text-[11.5px] tabular-nums text-subtle-foreground max-sm:hidden">{limitLine}</span>
<button type="submit" disabled={busy || uploading || (!text.trim() && !ready.length)} aria-label={t.send} className={cn(primaryBtn, 'max-sm:ml-auto max-sm:w-11 max-sm:px-0')}>
<Icon name="send" size={15} />
<span className="max-sm:sr-only">{t.send}</span>
</button>
</div>
<p role="status" className="px-2 text-[12.5px] text-success-foreground empty:hidden">
{sent && (
<span className="mmi-up mt-1 inline-flex items-center gap-1.5 pb-1">
<Icon name="check-circle" size={14} />
{sent}
</span>
)}
</p>
</form>
);
}
const sentLine = (t: Texts, n: number) => (n === 0 ? t.sentText : n === 1 ? t.sent1 : fill(t.sent, { n }));
/* ------------------------------------------------------------------ */
/* dropzone */
/* ------------------------------------------------------------------ */
function Dropzone({ t, title, description, defaultValue = '', attachments = [], maxFiles = 8, maxSizeMb = 20, support = [], note, onSubmit, className }: P) {
const f = useFormat();
const uid = useId();
const files = useAttachments(attachments, maxFiles, maxSizeMb, support, t);
const [text, setText] = useState(defaultValue);
const [drag, setDrag] = useState(0);
const [sent, setSent] = useSent();
const input = useRef<HTMLInputElement>(null);
const camera = useRef<HTMLInputElement>(null);
const accept = 'image/*,application/pdf,audio/*,video/*,.doc,.docx,.txt,.md,.csv,.xls,.xlsx';
const send = () => {
const ready = files.items.filter((a) => !a.error);
if (!text.trim() && !ready.length) return;
onSubmit?.({ text, attachments: ready });
setSent(sentLine(t, ready.length));
setText('');
files.setItems([]);
};
const drop = {
onDragEnter: (e: React.DragEvent) => (e.preventDefault(), setDrag((n) => n + 1)),
onDragOver: (e: React.DragEvent) => e.preventDefault(),
onDragLeave: () => setDrag((n) => Math.max(0, n - 1)),
onDrop: (e: React.DragEvent) => {
e.preventDefault();
setDrag(0);
if (e.dataTransfer.files.length) files.addFiles(e.dataTransfer.files);
},
};
const used = files.items.filter((a) => !a.error).reduce((s, a) => s + a.size, 0);
return (
<Stage title={title} description={description} className={className}>
<div ref={files.ref} {...drop} className="relative">
<input ref={input} id={`${uid}-f`} type="file" multiple accept={accept} className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
<input ref={camera} type="file" accept="image/*" capture="environment" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
{/* The zone: a large target from tablets on, two buttons on phones */}
<Reveal delay={40}>
<div className={cn('relative mb-3 hidden overflow-hidden rounded-2xl border-[1.5px] border-dashed p-6 text-center transition-colors duration-200 sm:block', drag ? 'border-primary bg-[color-mix(in_oklab,var(--primary)_8%,var(--card))]' : 'border-border-strong bg-[color-mix(in_oklab,var(--card)_60%,transparent)]')}>
<span aria-hidden className={cn('mx-auto grid size-12 place-items-center rounded-2xl border border-border bg-card text-foreground shadow-[var(--shadow-sm)] transition-transform duration-300 motion-reduce:transition-none', drag && '-translate-y-1 scale-105')}>
<Icon name={drag ? 'download' : 'cloud-upload'} size={22} />
</span>
<p className="mt-3 text-[15px] font-medium text-foreground">{drag ? t.drop : t.dropHint}</p>
<p className="mt-1 text-[12.5px] tabular-nums text-muted-foreground">{fill(t.limits, { count: files.items.length, max: maxFiles, mb: maxSizeMb })}</p>
<button type="button" onClick={() => input.current?.click()} className={cn(quietBtn, 'mt-4')}>
<Icon name="paperclip" size={15} />
{t.choose}
</button>
{support.length > 0 && (
<ul aria-label={t.reads} className="mx-auto mt-5 grid max-w-[38rem] grid-cols-2 gap-2 text-left md:grid-cols-4">
{support.map((s) => (
<li key={s.kind} className="rounded-xl border border-border bg-card p-3">
<p className="flex items-center gap-1.5 text-[12.5px] font-medium text-foreground">
<Icon name={KIND[s.kind].icon} size={14} style={{ color: `color-mix(in oklab, var(${KIND[s.kind].tone}) 75%, var(--foreground))` }} />
{t[s.kind]}
</p>
<p className="mt-1 text-[11.5px] leading-snug text-muted-foreground">{s.reads}</p>
<p className="mt-1.5 text-[11.5px] tabular-nums text-subtle-foreground">{[s.maxMb ? `${s.maxMb} MB` : null, s.maxSeconds ? fill(t.upTo, { v: f.time(s.maxSeconds) }) : null].filter(Boolean).join(' · ')}</p>
</li>
))}
</ul>
)}
</div>
<div className="mb-3 grid grid-cols-2 gap-2 sm:hidden">
<button type="button" onClick={() => camera.current?.click()} className={cn(quietBtn, 'h-12')}>
<Icon name="camera" size={16} />
{t.takePhoto}
</button>
<button type="button" onClick={() => input.current?.click()} className={cn(quietBtn, 'h-12')}>
<Icon name="paperclip" size={16} />
{t.choose}
</button>
</div>
</Reveal>
<Reveal delay={100}>
<Composer
t={t}
text={text}
setText={setText}
items={files.items}
remove={files.remove}
onSend={send}
sent={sent}
dropping={drag > 0}
limitLine={fill(t.total, { used: f.mb(used), max: maxSizeMb * maxFiles })}
tools={
<button type="button" aria-label={t.addFile} title={t.addFile} onClick={() => input.current?.click()} className={toolBtn}>
<Icon name="paperclip" size={18} />
</button>
}
/>
</Reveal>
{files.notice && (
<p role="alert" className="mmi-up mt-2 flex items-start gap-1.5 px-1 text-[12.5px] text-destructive">
<Icon name="alert-circle" size={14} className="mt-px shrink-0" />
{files.notice}
</p>
)}
<p className="mt-2 px-1 text-[12px] tabular-nums text-muted-foreground sm:hidden">{fill(t.limits, { count: files.items.length, max: maxFiles, mb: maxSizeMb })}</p>
{note && <Note>{note}</Note>}
</div>
</Stage>
);
}
/* ------------------------------------------------------------------ */
/* media-toolbar */
/* ------------------------------------------------------------------ */
function MediaToolbar({ t, title, description, defaultValue = '', attachments = [], maxFiles = 8, maxSizeMb = 20, support = [], screen, note, onSubmit, className }: P) {
const f = useFormat();
const files = useAttachments(attachments, maxFiles, maxSizeMb, support, t);
const [text, setText] = useState(defaultValue);
const [sent, setSent] = useSent();
const [rec, setRec] = useState<number | null>(null);
const [flash, setFlash] = useState(0);
const image = useRef<HTMLInputElement>(null);
const file = useRef<HTMLInputElement>(null);
const camera = useRef<HTMLInputElement>(null);
const shots = useRef(0);
const total = maxSizeMb * maxFiles;
const used = files.items.filter((a) => !a.error).reduce((s, a) => s + a.size, 0);
const share = Math.min(100, (used / (total * 1e6)) * 100);
const supported = (k: MediaKind) => !support.length || support.some((s) => s.kind === k);
useEffect(() => {
if (rec === null || prefersReducedMotion()) return;
const id = window.setInterval(() => setRec((r) => (r === null ? r : r + 1)), 1000);
return () => window.clearInterval(id);
}, [rec === null]); // eslint-disable-line react-hooks/exhaustive-deps
const tools: { id: string; icon: IconName; label: string; kind: MediaKind; run: () => void }[] = [
{ id: 'image', icon: 'image', label: t.addImage, kind: 'image', run: () => image.current?.click() },
{ id: 'file', icon: 'paperclip', label: t.addFile, kind: 'pdf', run: () => file.current?.click() },
{ id: 'camera', icon: 'camera', label: t.useCamera, kind: 'image', run: () => camera.current?.click() },
{
id: 'screen',
icon: 'monitor',
label: t.captureScreen,
kind: 'screenshot',
run: () => {
setFlash((n) => n + 1);
files.add([{ name: `${t.screenshot} ${++shots.current}.png`, kind: 'screenshot', size: 840_000 + shots.current * 61_000, media: screen }]);
},
},
{
id: 'voice',
icon: 'mic',
label: t.recordVoice,
kind: 'audio',
run: () => {
if (rec === null) return setRec(0);
files.add([{ name: `${t.voiceNote}.m4a`, kind: 'audio', size: 24_000 * Math.max(1, rec), duration: Math.max(1, rec) }]);
setRec(null);
},
},
];
const send = () => {
const ready = files.items.filter((a) => !a.error);
if (!text.trim() && !ready.length) return;
onSubmit?.({ text, attachments: ready });
setSent(sentLine(t, ready.length));
setText('');
files.setItems([]);
};
return (
<Stage title={title} description={description} className={className}>
<div ref={files.ref}>
<input ref={image} type="file" multiple accept="image/*" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
<input ref={file} type="file" multiple accept="application/pdf,.doc,.docx,.txt,.md,.csv,.xls,.xlsx,audio/*,video/*" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
<input ref={camera} type="file" accept="image/*" capture="environment" className="sr-only" tabIndex={-1} onChange={(e) => (e.target.files && files.addFiles(e.target.files), (e.target.value = ''))} />
{flash > 0 && <span key={flash} aria-hidden className="mmi-flash pointer-events-none fixed inset-0 z-50 bg-white opacity-0" />}
<Reveal delay={60}>
<Composer
t={t}
text={text}
setText={setText}
items={files.items}
remove={files.remove}
onSend={send}
sent={sent}
busy={rec !== null}
limitLine={fill(t.limits, { count: files.items.length, max: maxFiles, mb: maxSizeMb })}
tools={
<div role="group" aria-label={t.attachments} className="flex items-center gap-0.5">
{tools.map((x) =>
x.id === 'voice' && rec !== null ? (
<button key={x.id} type="button" onClick={x.run} className={cn(btn, 'gap-2 bg-[color-mix(in_oklab,var(--destructive)_12%,transparent)] px-2.5 text-destructive hover:bg-[color-mix(in_oklab,var(--destructive)_18%,transparent)]')}>
<span aria-hidden className="mmi-blink size-2 rounded-full bg-destructive" />
<span className="tabular-nums">{f.time(rec)}</span>
<span className="sr-only">{t.stop}</span>
<Icon name="stop" size={13} />
</button>
) : (
<button key={x.id} type="button" aria-label={x.label} title={x.label} disabled={!supported(x.kind) || rec !== null} onClick={x.run} className={cn(toolBtn, x.id === 'camera' && 'sm:hidden', x.id === 'screen' && 'max-sm:hidden')}>
<Icon name={x.icon} size={18} />
</button>
),
)}
</div>
}
/>
</Reveal>
{files.notice && (
<p role="alert" className="mmi-up mt-2 flex items-start gap-1.5 px-1 text-[12.5px] text-destructive">
<Icon name="alert-circle" size={14} className="mt-px shrink-0" />
{files.notice}
</p>
)}
{/* What the assistant reads, with the limits per kind */}
<Reveal delay={140} className="mt-4 rounded-2xl border border-border bg-[color-mix(in_oklab,var(--card)_70%,transparent)] p-4">
<div className="flex flex-wrap items-center justify-between gap-x-4 gap-y-2">
<p className="text-[13px] font-medium text-foreground">{t.reads}</p>
<div className="flex min-w-[12rem] items-center gap-2.5 max-sm:w-full">
<span role="meter" aria-label={t.total.replace('{used}', f.mb(used)).replace('{max}', String(total))} aria-valuemin={0} aria-valuemax={100} aria-valuenow={Math.round(share)} className="block h-1.5 flex-1 overflow-hidden rounded-full bg-track">
<span className={cn('block h-full origin-left rounded-full transition-transform duration-500', share > 85 ? 'bg-warning' : 'bg-primary')} style={{ transform: `scaleX(${round(share / 100, 3)})` }} />
</span>
<span className="text-[12px] tabular-nums text-muted-foreground">{fill(t.total, { used: f.mb(used), max: total })}</span>
</div>
</div>
<ul className="mt-3 grid grid-cols-1 gap-x-6 gap-y-2.5 sm:grid-cols-2">
{support.map((s) => (
<li key={s.kind} className="flex items-start gap-2.5">
<span aria-hidden className="grid size-8 shrink-0 place-items-center rounded-lg" style={{ background: `color-mix(in oklab, var(${KIND[s.kind].tone}) 14%, var(--card))`, color: `color-mix(in oklab, var(${KIND[s.kind].tone}) 75%, var(--foreground))` }}>
<Icon name={KIND[s.kind].icon} size={15} />
</span>
<span className="min-w-0">
<span className="flex flex-wrap items-baseline gap-x-2 text-[13px] font-medium text-foreground">
{t[s.kind]}
<span className="text-[11.5px] font-normal tabular-nums text-subtle-foreground">{[s.maxMb ? `${s.maxMb} MB` : null, s.maxSeconds ? fill(t.upTo, { v: f.time(s.maxSeconds) }) : null].filter(Boolean).join(' · ')}</span>
</span>
<span className="block text-[12.5px] leading-snug text-muted-foreground">{s.reads}</span>
</span>
</li>
))}
</ul>
</Reveal>
{note && <Note>{note}</Note>}
</div>
</Stage>
);
}
/* ------------------------------------------------------------------ */
/* voice-note */
/* ------------------------------------------------------------------ */
const BARS = 32;
/** A calm, believable input level: two slow waves and a little noise */
function levelAt(i: number, tick: number) {
const v = 0.35 + 0.3 * Math.sin(tick / 3 + i / 2.4) + 0.22 * Math.sin(tick / 1.7 + i * 1.3) + 0.12 * Math.sin(tick * 1.9 + i * 3.1);
return round(Math.max(0.08, Math.min(1, v)), 2);
}
function VoiceNote({ t, title, description, defaultValue = '', maxSeconds = 300, transcript = '', autoStart, note, onSubmit, className }: P) {
const f = useFormat();
const uid = useId();
const [state, setState] = useState<'idle' | 'recording' | 'paused' | 'done'>('idle');
const [secs, setSecs] = useState(0);
const [tick, setTick] = useState(0);
const [peaks, setPeaks] = useState<number[]>([]);
const [mode, setMode] = useState<'audio' | 'text'>('audio');
const [text, setText] = useState(defaultValue);
const [sent, setSent] = useSent();
const [ref, inView] = useInView<HTMLDivElement>();
const started = useRef(false);
const words = transcript.split(/(\s+)/);
// About two words a second appear while recording, all of them once it stops
const heard = state === 'idle' ? 0 : state === 'done' ? words.length : Math.min(words.length, secs * 4);
useEffect(() => {
if (autoStart && inView && !started.current && !prefersReducedMotion()) {
started.current = true;
setState('recording');
}
}, [autoStart, inView]);
useEffect(() => {
if (state !== 'recording') return;
if (prefersReducedMotion()) {
setSecs((s) => Math.max(s, 14));
return;
}
const a = window.setInterval(() => setTick((n) => n + 1), 110);
const b = window.setInterval(() => setSecs((s) => (s + 1 >= maxSeconds ? (setState('done'), maxSeconds) : s + 1)), 1000);
return () => (window.clearInterval(a), window.clearInterval(b));
}, [state, maxSeconds]);
useEffect(() => {
if (state === 'recording' && tick % 3 === 0) setPeaks((p) => [...p.slice(-47), levelAt(0, tick)]);
}, [tick, state]);
const reset = () => (setState('idle'), setSecs(0), setPeaks([]), setTick(0));
const levels = Array.from({ length: BARS }, (_, i) => (state === 'recording' ? levelAt(i, tick) : state === 'paused' ? 0.1 : levelAt(i, 7) * 0.5));
const share = Math.min(100, (secs / maxSeconds) * 100);
const wave = peaks.length ? peaks : Array.from({ length: 48 }, (_, i) => levelAt(i, 4));
const send = () => {
const att: MediaAttachment[] = state === 'done' && mode === 'audio' ? [{ id: 'voice', name: `${t.voiceNote}.m4a`, kind: 'audio', size: secs * 24_000, duration: secs }] : [];
const body = state === 'done' && mode === 'text' ? [text, transcript.replace(/~/g, '')].filter(Boolean).join(' ') : text;
if (!body.trim() && !att.length) return;
onSubmit?.({ text: body, attachments: att });
setSent(sentLine(t, att.length));
setText('');
reset();
};
return (
<Stage title={title} description={description} className={className}>
<div ref={ref}>
<Reveal effect="scale" className={cn(card, 'overflow-hidden p-0')}>
{/* Recorder */}
<div className={cn('relative px-4 pb-5 pt-6 text-center transition-colors duration-300 sm:px-6', state === 'recording' && 'bg-[color-mix(in_oklab,var(--destructive)_5%,var(--card))]')}>
<p className="flex items-center justify-center gap-2 text-[13px] font-medium" aria-live="polite">
{state === 'recording' && <span aria-hidden className="mmi-blink size-2 rounded-full bg-destructive" />}
<span className={state === 'recording' ? 'text-destructive' : 'text-muted-foreground'}>{state === 'recording' ? t.recording : state === 'paused' ? t.paused : state === 'done' ? t.voiceNote : t.record}</span>
</p>
<p className="mt-1 text-[34px] font-semibold tabular-nums tracking-[-0.03em] text-foreground sm:text-[40px]">{f.time(secs)}</p>
<p className="text-[12px] tabular-nums text-muted-foreground">{fill(t.left, { v: f.time(Math.max(0, maxSeconds - secs)) })}</p>
{state === 'done' ? (
<div className="mmi-up mx-auto mt-5 flex max-w-[28rem] items-center gap-3 rounded-xl border border-border bg-surface p-2.5 text-left">
<span aria-hidden className="grid size-10 shrink-0 place-items-center rounded-lg bg-[color-mix(in_oklab,var(--warning)_15%,var(--card))] text-warning-foreground">
<Icon name="mic" size={17} />
</span>
<svg aria-hidden viewBox={`0 0 ${wave.length * 4} 32`} preserveAspectRatio="none" className="h-8 min-w-0 flex-1">
{wave.map((v, i) => (
<rect key={i} x={i * 4 + 0.75} y={round(16 - v * 14)} width="2.5" height={round(Math.max(2, v * 28))} rx="1.25" fill="color-mix(in oklab, var(--foreground) 55%, transparent)" />
))}
</svg>
<span className="shrink-0 text-[12px] tabular-nums text-muted-foreground">{f.time(secs)}</span>
</div>
) : (
<div role="meter" aria-label={t.level} aria-valuemin={0} aria-valuemax={100} aria-valuenow={Math.round((levels.reduce((s, v) => s + v, 0) / BARS) * 100)} className="mx-auto mt-5 flex h-16 max-w-[28rem] items-center justify-center gap-[3px]">
{levels.map((v, i) => (
<span key={i} aria-hidden className={cn('w-[5px] rounded-full transition-[height] duration-100 ease-linear motion-reduce:transition-none', state === 'recording' ? 'bg-destructive' : 'bg-[color-mix(in_oklab,var(--foreground)_22%,transparent)]')} style={{ height: `${round(8 + v * 56)}px`, opacity: state === 'recording' ? round(0.45 + v * 0.55, 2) : 1 }} />
))}
</div>
)}
<div aria-hidden className="mx-auto mt-4 h-1 max-w-[28rem] overflow-hidden rounded-full bg-track">
<span className="block h-full origin-left rounded-full bg-destructive transition-transform duration-1000 ease-linear" style={{ transform: `scaleX(${round(share / 100, 3)})` }} />
</div>
<div className="mt-5 flex flex-wrap items-center justify-center gap-2">
{state === 'idle' && (
<button type="button" onClick={() => setState('recording')} className="grid size-16 place-items-center rounded-full bg-destructive text-destructive-foreground shadow-[0_6px_20px_color-mix(in_oklab,var(--destructive)_35%,transparent)] transition-transform hover:scale-105 active:scale-95 motion-reduce:transition-none" aria-label={t.record}>
<Icon name="mic" size={24} />
</button>
)}
{(state === 'recording' || state === 'paused') && (
<>
<button type="button" onClick={reset} className={quietBtn}>
<Icon name="trash" size={15} />
{t.discard}
</button>
<button type="button" onClick={() => setState(state === 'recording' ? 'paused' : 'recording')} className={quietBtn}>
<Icon name={state === 'recording' ? 'pause' : 'play'} size={15} />
{state === 'recording' ? t.pause : t.resume}
</button>
<button type="button" onClick={() => setState('done')} className={primaryBtn}>
<Icon name="stop" size={14} />
{t.stop}
</button>
</>
)}
{state === 'done' && (
<>
<button type="button" onClick={reset} className={quietBtn}>
<Icon name="trash" size={15} />
{t.discard}
</button>
<div role="radiogroup" aria-label={t.voiceNote} className="flex gap-1 rounded-lg bg-[color-mix(in_oklab,var(--foreground)_6%,transparent)] p-1">
{(['audio', 'text'] as const).map((m) => (
<button key={m} type="button" role="radio" aria-checked={mode === m} tabIndex={mode === m ? 0 : -1} onClick={() => setMode(m)} onKeyDown={(e) => ['ArrowLeft', 'ArrowRight'].includes(e.key) && (e.preventDefault(), setMode(m === 'audio' ? 'text' : 'audio'), (e.currentTarget.parentElement?.querySelector(`[data-m="${m === 'audio' ? 'text' : 'audio'}"]`) as HTMLElement | null)?.focus())} data-m={m} className={cn('inline-flex h-11 items-center gap-1.5 rounded-md px-3 text-[12.5px] font-medium transition-colors sm:h-8', mode === m ? 'bg-card text-foreground shadow-[var(--highlight),var(--shadow-sm)]' : 'text-muted-foreground hover:text-foreground')}>
<Icon name={m === 'audio' ? 'volume' : 'type'} size={14} />
{m === 'audio' ? t.asAudio : t.asText}
</button>
))}
</div>
</>
)}
</div>
</div>
{/* Transcript as it is heard */}
{transcript && state !== 'idle' && (
<div className="border-t border-border px-4 py-4 sm:px-6">
<p className="text-[12px] font-medium text-muted-foreground">{t.transcript}</p>
<p className="mt-1.5 text-pretty text-[14.5px] leading-relaxed text-foreground" aria-live="polite">
{words.slice(0, heard).map((w, i) =>
w.startsWith('~') ? (
<span key={i} className="underline decoration-[color-mix(in_oklab,var(--warning)_80%,transparent)] decoration-dotted decoration-2 underline-offset-4">
{w.replace(/~/g, '')}
</span>
) : (
<span key={i}>{w.replace(/~/g, '')}</span>
),
)}
{state === 'recording' && <span aria-hidden className="mmi-blink ml-0.5 inline-block h-4 w-0.5 translate-y-0.5 bg-foreground" />}
</p>
{transcript.includes('~') && <p className="mt-2 text-[12px] text-muted-foreground">{t.uncertain}</p>}
</div>
)}
{/* Message */}
<form
onSubmit={(e) => {
e.preventDefault();
send();
}}
className="flex items-end gap-2 border-t border-border p-2"
>
<label htmlFor={`${uid}-m`} className="sr-only">
{t.message}
</label>
<textarea id={`${uid}-m`} rows={1} value={text} onChange={(e) => setText(e.target.value)} placeholder={t.placeholder} className="block min-h-11 w-0 min-w-0 flex-1 resize-none bg-transparent px-2.5 py-2.5 text-[15px] leading-snug text-foreground [field-sizing:content] placeholder:text-subtle-foreground focus-visible:outline-none" />
<button type="submit" disabled={state === 'recording' || state === 'paused' || (!text.trim() && state !== 'done')} aria-label={t.send} className={cn(primaryBtn, 'max-sm:w-11 max-sm:px-0')}>
<Icon name="send" size={15} />
<span className="max-sm:sr-only">{t.send}</span>
</button>
</form>
<p role="status" className="px-4 pb-3 text-[12.5px] text-success-foreground empty:hidden">
{sent && (
<span className="mmi-up inline-flex items-center gap-1.5">
<Icon name="check-circle" size={14} />
{sent}
</span>
)}
</p>
</Reveal>
<Note icon="mic">{note ?? t.micNote}</Note>
</div>
</Stage>
);
}
/* ------------------------------------------------------------------ */
/* camera-capture */
/* ------------------------------------------------------------------ */
type Box = { x: number; y: number; w: number; h: number };
function CameraCapture({ t, title, description, defaultValue = '', preview, screen, windows = [], maxFiles = 8, maxSizeMb = 20, support = [], note, onSubmit, className }: P) {
const uid = useId();
const files = useAttachments([], maxFiles, maxSizeMb, support, t);
const [mode, setMode] = useState<'photo' | 'screen'>('photo');
const [shot, setShot] = useState(false);
const [flash, setFlash] = useState(0);
const [box, setBox] = useState<Box | null>(null);
const [win, setWin] = useState(windows[0]?.id ?? '');
const [text, setText] = useState(defaultValue);
const [sent, setSent] = useSent();
const [front, setFront] = useState(false);
const frame = useRef<HTMLDivElement>(null);
const start = useRef<{ x: number; y: number } | null>(null);
const seq = useRef(0);
const pic = mode === 'photo' ? preview : screen;
const pct = (e: React.PointerEvent) => {
const r = frame.current!.getBoundingClientRect();
return { x: round(Math.max(0, Math.min(100, ((e.clientX - r.left) / r.width) * 100)), 1), y: round(Math.max(0, Math.min(100, ((e.clientY - r.top) / r.height) * 100)), 1) };
};
const capture = () => {
setFlash((n) => n + 1);
setShot(true);
setBox(null);
};
const attach = () => {
const name = mode === 'photo' ? `${t.photo} ${++seq.current}.jpg` : `${windows.find((w) => w.id === win)?.label ?? t.screen}.png`;
files.add([{ name: box ? `${name.replace(/\.\w+$/, '')} · ${t.area}${mode === 'photo' ? '.jpg' : '.png'}` : name, kind: mode === 'photo' ? 'image' : 'screenshot', size: mode === 'photo' ? 2_400_000 : 910_000, media: pic }]);
setShot(false);
setBox(null);
};
const send = () => {
const ready = files.items.filter((a) => !a.error);
if (!text.trim() && !ready.length) return;
onSubmit?.({ text, attachments: ready });
setSent(sentLine(t, ready.length));
setText('');
files.setItems([]);
};
const modes = [
{ id: 'photo' as const, label: t.camera, icon: 'camera' as IconName },
{ id: 'screen' as const, label: t.screen, icon: 'monitor' as IconName },
];
return (
<Stage title={title} description={description} className={className} wide>
<div ref={files.ref} className="grid grid-cols-1 gap-4 lg:grid-cols-[minmax(0,1fr)_22rem] lg:items-start lg:gap-6">
<Reveal effect="scale" className="min-w-0">
<div className="dark rounded-2xl border border-border bg-bezel p-2 shadow-[var(--shadow-lg)]">
<div className="flex items-center justify-between gap-2 px-1 pb-2">
<div role="radiogroup" aria-label={t.capture} className="flex gap-1 rounded-lg bg-[rgb(255_255_255/0.08)] p-1">
{modes.map((m) => (
<button key={m.id} type="button" role="radio" aria-checked={mode === m.id} tabIndex={mode === m.id ? 0 : -1} data-m={m.id} onClick={() => (setMode(m.id), setShot(false), setBox(null))} onKeyDown={(e) => ['ArrowLeft', 'ArrowRight'].includes(e.key) && (e.preventDefault(), setMode(m.id === 'photo' ? 'screen' : 'photo'), setShot(false), setBox(null), (e.currentTarget.parentElement?.querySelector(`[data-m="${m.id === 'photo' ? 'screen' : 'photo'}"]`) as HTMLElement | null)?.focus())} className={cn('inline-flex h-11 items-center gap-1.5 rounded-md px-3 text-[12.5px] font-medium transition-colors sm:h-8', mode === m.id ? 'bg-white text-[#111]' : 'text-white/75 hover:text-white')}>
<Icon name={m.icon} size={14} />
{m.label}
</button>
))}
</div>
{mode === 'photo' && !shot && (
<button type="button" aria-label={t.flip} title={t.flip} onClick={() => setFront((v) => !v)} className="grid size-11 place-items-center rounded-lg text-white/80 hover:bg-white/10 hover:text-white sm:size-9">
<Icon name="refresh" size={17} />
</button>
)}
</div>
{mode === 'screen' && !shot && windows.length > 0 && (
<div
role="radiogroup"
aria-label={t.pickWindow}
// APG radio group: one tab stop, the arrow keys move and choose
onKeyDown={(e) => {
const step = e.key === 'ArrowRight' || e.key === 'ArrowDown' ? 1 : e.key === 'ArrowLeft' || e.key === 'ArrowUp' ? -1 : 0;
if (!step) return;
e.preventDefault();
const i = windows.findIndex((w) => w.id === win);
const next = windows[(i + step + windows.length) % windows.length];
setWin(next.id);
(e.currentTarget.querySelector('[data-w="' + next.id + '"]') as HTMLElement | null)?.focus();
}}
className="kit-no-scrollbar mb-2 flex gap-1.5 overflow-x-auto px-1"
>
{windows.map((w) => (
<button key={w.id} type="button" role="radio" aria-checked={win === w.id} tabIndex={win === w.id ? 0 : -1} data-w={w.id} onClick={() => setWin(w.id)} className={cn('inline-flex h-11 shrink-0 items-center gap-1.5 rounded-lg border px-3 text-[12.5px] font-medium transition-colors sm:h-9', win === w.id ? 'border-white/70 bg-white/12 text-white' : 'border-white/15 text-white/70 hover:text-white')}>
<Icon name={w.icon ?? 'layout'} size={14} />
{w.label}
</button>
))}
</div>
)}
<div
ref={frame}
role={shot ? 'group' : 'img'}
aria-label={shot ? t.markArea : t.viewfinder}
className={cn('relative isolate aspect-[4/3] overflow-hidden rounded-xl bg-black', shot && 'cursor-crosshair touch-none select-none')}
onPointerDown={(e) => {
if (!shot || e.button !== 0) return;
e.currentTarget.setPointerCapture(e.pointerId);
start.current = pct(e);
setBox({ ...start.current, w: 0, h: 0 });
}}
onPointerMove={(e) => {
if (!start.current) return;
const p = pct(e);
const s = start.current;
setBox({ x: Math.min(s.x, p.x), y: Math.min(s.y, p.y), w: round(Math.abs(p.x - s.x), 1), h: round(Math.abs(p.y - s.y), 1) });
}}
onPointerUp={() => {
start.current = null;
setBox((b) => (b && b.w > 3 && b.h > 3 ? b : null));
}}
>
<div data-clip-ok className={cn('absolute inset-0 [&>*]:size-full', mode === 'photo' && front && !shot && '-scale-x-100')}>
{pic}
</div>
{!shot && mode === 'photo' && (
<div aria-hidden className="pointer-events-none absolute inset-6">
{['left-0 top-0 border-l-2 border-t-2 rounded-tl-lg', 'right-0 top-0 border-r-2 border-t-2 rounded-tr-lg', 'bottom-0 left-0 border-b-2 border-l-2 rounded-bl-lg', 'bottom-0 right-0 border-b-2 border-r-2 rounded-br-lg'].map((c) => (
<span key={c} className={cn('absolute size-7 border-white/85', c)} />
))}
</div>
)}
{box && (
<>
<div aria-hidden className="pointer-events-none absolute inset-0 bg-black/45" style={{ clipPath: `polygon(0 0,100% 0,100% 100%,0 100%,0 0,${box.x}% ${box.y}%,${box.x}% ${box.y + box.h}%,${box.x + box.w}% ${box.y + box.h}%,${box.x + box.w}% ${box.y}%,${box.x}% ${box.y}%)`, clipRule: 'evenodd' }} />
<div aria-hidden className="pointer-events-none absolute rounded-md border-2 border-white shadow-[0_0_0_1px_rgb(0_0_0/0.4)]" style={{ left: `${box.x}%`, top: `${box.y}%`, width: `${box.w}%`, height: `${box.h}%` }} />
</>
)}
{shot && !box && (
<span className="pointer-events-none absolute inset-x-0 bottom-3 mx-auto w-fit max-w-[90%] rounded-lg bg-black/65 px-3 py-1.5 text-center text-[12.5px] font-medium text-white backdrop-blur-sm">
<Icon name="crop" size={13} className="mr-1.5 inline align-[-2px]" />
{t.markArea}
</span>
)}
{flash > 0 && <span key={flash} aria-hidden className="mmi-flash pointer-events-none absolute inset-0 bg-white opacity-0" />}
</div>
<div className="flex items-center justify-center gap-3 px-1 pb-1 pt-3">
{shot ? (
<>
<button type="button" onClick={() => (setShot(false), setBox(null))} className="inline-flex h-11 items-center gap-1.5 rounded-lg border border-white/20 px-3.5 text-[13px] font-medium text-white hover:bg-white/10">
<Icon name="rotate-ccw" size={15} />
{t.retake}
</button>
{box && (
<button type="button" onClick={() => setBox(null)} className="inline-flex h-11 items-center gap-1.5 rounded-lg border border-white/20 px-3.5 text-[13px] font-medium text-white hover:bg-white/10">
{t.wholePicture}
</button>
)}
<button type="button" onClick={attach} className="inline-flex h-11 items-center gap-1.5 rounded-lg bg-white px-4 text-[13px] font-semibold text-[#111] hover:bg-white/90">
<Icon name="paperclip" size={15} />
{t.attach}
</button>
</>
) : (
<button type="button" aria-label={mode === 'photo' ? t.shutter : t.capture} onClick={capture} className="grid size-16 place-items-center rounded-full border-4 border-white/85 transition-transform hover:scale-105 active:scale-95 motion-reduce:transition-none">
<span className="size-11 rounded-full bg-white" />
</button>
)}
</div>
</div>
</Reveal>
<Reveal delay={120} className="lg:sticky lg:top-6">
<Composer
t={t}
text={text}
setText={setText}
items={files.items}
remove={files.remove}
onSend={send}
sent={sent}
limitLine={undefined}
tools={
<span className="inline-flex items-center gap-1.5 px-1.5 text-[12px] tabular-nums text-muted-foreground" id={`${uid}-n`}>
<Icon name="paperclip" size={14} />
{files.items.length}/{maxFiles}
</span>
}
/>
<Note>{note ?? t.cameraNote}</Note>
<ol className="mt-5 space-y-3 border-t border-border pt-5 max-lg:hidden">
{t.howTo.split('|').map((x, i) => (
<li key={x} className="flex items-center gap-3 text-[13.5px] text-foreground">
<span aria-hidden className="grid size-7 shrink-0 place-items-center rounded-full border border-border bg-card text-[12px] font-semibold tabular-nums text-muted-foreground">{i + 1}</span>
{x}
</li>
))}
</ol>
</Reveal>
</div>
</Stage>
);
}
/* ------------------------------------------------------------------ */
/**
* Input for an AI that reads more than text: a drop zone with the limits per
* kind, a toolbar for image, file, camera, screenshot and voice with what the
* assistant reads, a voice note with live level, timer and transcript, and a
* camera or screen capture where you mark the part that matters. Chips show
* upload progress, size and errors. Text features (commands, mentions) are
* prompt-input.
*/
export function MultimodalInput({ variant = 'dropzone', labels, ...props }: MultimodalInputProps) {
const t = useTexts(TEXTS, labels);
const p: P = { ...props, t };
switch (variant) {
case 'media-toolbar':
return <MediaToolbar {...p} />;
case 'voice-note':
return <VoiceNote {...p} />;
case 'camera-capture':
return <CameraCapture {...p} />;
default:
return <Dropzone {...p} />;
}
}Häufige Fragen
Wie installiere ich Multimodale Eingabe?
Mit aiw add @ai-website/multimodal-input. Der Befehl legt die Datei components/ui/multimodal-input.tsx in dein Projekt, der Code gehört danach dir. Mit dabei sind icon und reveal. Weitere npm-Pakete braucht sie nicht.
Welche Varianten hat Multimodale Eingabe?
4 Varianten, Ablagezone, Medienleiste, Sprachnachricht und Kamera und Bildschirm. Du wählst sie mit der Prop variant, zum Beispiel variant="media-toolbar".
Kann ich die Texte von Multimodale Eingabe übersetzen?
Ja. Multimodale Eingabe bringt 67 eigene Texte auf Deutsch und Englisch mit, etwa „Nachricht“, „Frag zu deinen Dateien, Fotos oder einer Aufnahme“ und „Senden“. Die Sprache folgt dem KitProvider, einzelne Texte ersetzt du über die Prop labels.
Kann ein KI-Assistent Multimodale Eingabe einbauen?
Ja. Über den MCP-Server der aiw CLI finden Claude, Cursor, GitHub Copilot, Windsurf, Codex und Gemini Multimodale Eingabe mit aiw_components und installieren sie mit aiw_add @ai-website/multimodal-input. Nenn im Prompt die Variante, etwa variant="dropzone", dann schreibt der Assistent deine Inhalte direkt in die Props.