Text - Recursive Split

Type identifier: text:recursiveSplit Category: Text Operations

Description

The Text - Recursive Split node divides text into smaller chunks suitable for processing by language models or other text analysis tools. It splits on natural boundaries (paragraphs, lines, words) where possible, while respecting the configured chunk size. Overlapping between chunks ensures context is preserved across boundaries.

Input Handles

Output Handles

Configuration Options

Behaviour

1. Accepts string or Content input (Content is serialised to plain text)

2. Recursively splits text using a hierarchy of separators:

   • First attempts to split on paragraph breaks (\n\n)
   • Falls back to line breaks (\n)
   • Then word boundaries (space)
   • Finally character-by-character if needed
3. Merges split segments into chunks respecting the size limit
4. Maintains overlap between consecutive chunks for context continuity
5. Returns non-empty, trimmed chunks

Splitting Algorithm

The recursive splitter prioritises preserving semantic units:

1. Paragraph Split: Tries to keep paragraphs intact
2. Line Split: Breaks on line boundaries if paragraphs are too large
3. Word Split: Splits at word boundaries for longer lines
4. Character Split: Last resort for very long words

Overlap Behaviour

Overlap ensures context continuity:

• The end of chunk N overlaps with the beginning of chunk N+1
• Overlap size cannot exceed chunk size
• Overlap is measured in characters

Examples

Basic Text Chunking

Configuration:

• Chunk Size: 100
• Chunk Overlap: 10

Input:

This is a longer text that needs to be split into chunks.

It has multiple paragraphs.

Each paragraph contains different information.

Output (chunks):

[
  "This is a longer text that needs to be split into chunks.",
  "It has multiple paragraphs.",
  "Each paragraph contains different information."
]

Document Preparation for LLM

Configuration:

• Chunk Size: 750
• Chunk Overlap: 20

Use case: Preparing a long document for embedding or analysis by an AI model.