Sobmit Docs

Embed Data node

The Embed Data node embeds incoming text into vector form and upserts the vectors into the flow’s vector namespace (an Upstash Vector index named after the flow id). A Search Data node — or an Agent’s vector-search tool — can then retrieve them by semantic similarity.

Sources

  • Input handle — the main payload flowing through the chain (wire it from the Start node).
  • Documents handle — extra floating sources (e.g. an Upload or Prompt node).

Both resolve identically:

  • A string passes verbatim.
  • A text upload contributes its content (txt / markdown / CSV).
  • Excel files (.xlsx, .xls, .xlsm, .ods) are parsed into tab-separated text before chunking, so spreadsheet data is searchable too.
  • PDF files are extracted with unpdf (a serverless-safe PDF.js wrapper).
  • JSON shapes are serialized; images, video and audio (no readable text) are skipped.

To embed a document, wire Start → Embed Data (its Input) so the node is reachable — the engine walks from Start and never executes an unreachable node.

Configuration

  • Model — the embedding model. Defaults to qwen3-embedding-8b. The Search Data node (and an Agent’s vector tool) must use the SAME model per flow, or queries return nothing.
  • Chunk size — target characters per chunk (default 8000, clamped 200–12000).
  • Chunk overlap — characters shared between consecutive chunks (default 200), so context survives a split.
  • Knowledge base — optional name for this node’s own index (<flowId>:<name>, e.g. handbook). Use it to keep several document sets in one flow apart: an Embed Data node and a Search Data or Chat node that carry the SAME name share that index, and two Embed Data nodes with the same name feed the same one. Leave it empty for the flow’s shared index.

Chunking

chunkText splits recursively, preferring paragraph / line / sentence / word seams over a hard mid-token cut. A single node stores at most 500 chunks; a source that would exceed it fails with a clear message instead of spawning unbounded (and billed) embeddings.

Output

The node returns { embedded } — the number of chunks now stored in the flow’s namespace for this node. The card shows “N chunks embedded” after a run.

Notes

  • Requires a saved flow (the flow id is the default namespace; a knowledge base name adds <flowId>:<name>).
  • Requires UPSTASH_VECTOR_REST_URL and UPSTASH_VECTOR_REST_TOKEN; otherwise the step fails with a clear message.
  • Re-running this node replaces its stored chunks (it clears its own vectors first), so re-embedding the same or an edited document never leaves duplicate results in Search Data. Different Embed Data nodes still accumulate in the shared index.

Related