Sobmit Docs

Embed Data node

The Embed Data node embeds incoming text into vector form and upserts the vectors into the flow’s vector namespace (an Upstash Vector index named after the flow id). A Search Data node — or an Agent’s vector-search tool — can then retrieve them by semantic similarity.

Sources

  • Input handle — the main payload flowing through the chain (wire it from the Start node).
  • Documents handle — extra floating sources (e.g. an Upload or Prompt node).

Both resolve identically:

  • A string passes verbatim.
  • A text upload contributes its content (txt / markdown / CSV).
  • Excel files (.xlsx, .xls, .xlsm, .ods) are parsed into tab-separated text before chunking, so spreadsheet data is searchable too.
  • PDF files are extracted with unpdf (a serverless-safe PDF.js wrapper).
  • JSON shapes are serialized; images, video and audio (no readable text) are skipped.

To embed a document, wire Start → Embed Data (its Input) so the node is reachable — the engine walks from Start and never executes an unreachable node.

Configuration

  • Model — the embedding model. Defaults to openai/text-embedding-3-small. The Search Data node (and an Agent’s vector tool) must use the SAME model per flow, or queries return nothing.
  • Chunk size — target characters per chunk (default 8000, clamped 200–12000).
  • Chunk overlap — characters shared between consecutive chunks (default 200), so context survives a split.

Chunking

chunkText splits recursively, preferring paragraph / line / sentence / word seams over a hard mid-token cut. A single node stores at most 500 chunks; a source that would exceed it fails with a clear message instead of spawning unbounded (and billed) embeddings.

Output

The node returns { embedded } — the number of chunks now stored in the flow’s namespace for this node. The card shows “N chunks embedded” after a run.

Notes

  • Requires a saved flow (the flow id is the namespace).
  • Requires UPSTASH_VECTOR_REST_URL and UPSTASH_VECTOR_REST_TOKEN; otherwise the step fails with a clear message.
  • Chunk ids are deterministic (<nodeId>:<index>), so a replayed step overwrites rather than duplicates.

Related