Package com.hw.langchain.text.splitter

Class RecursiveCharacterTextSplitter

java.lang.Object

com.hw.langchain.text.splitter.TextSplitter

com.hw.langchain.text.splitter.RecursiveCharacterTextSplitter

All Implemented Interfaces:

BaseDocumentTransformer

public class RecursiveCharacterTextSplitter
extends TextSplitter

Implementation of splitting text that looks at characters. Recursively tries to split by different characters to find one that works.

Author:

HamaWhite

Field Summary

Fields

Modifier and Type	Field	Description
protected List<String>	separators

Fields inherited from class com.hw.langchain.text.splitter.TextSplitter

addStartIndex, chunkOverlap, chunkSize, keepSeparator, lengthFunction

Constructor Summary

Constructors

Constructor	Description
RecursiveCharacterTextSplitter()

Method Summary

Modifier and Type	Method	Description
List<String>	innerSplitText(String text, List<String> separators)	Split incoming text and return chunks.
List<String>	splitText(String text)	Split text into multiple components.

Methods inherited from class com.hw.langchain.text.splitter.TextSplitter

createDocuments, mergeSplits, splitDocuments, transformDocuments

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

separators

protected List<String> separators

Constructor Details

RecursiveCharacterTextSplitter

public RecursiveCharacterTextSplitter()

Method Details

innerSplitText

public List<String> innerSplitText(String text,
 List<String> separators)

Split incoming text and return chunks.

splitText

public List<String> splitText(String text)

Description copied from class: TextSplitter

Split text into multiple components.

Specified by:

splitText in class TextSplitter