org.apache.lucene.analysis.ngram

Class EdgeNGramTokenizer

public class EdgeNGramTokenizer extends Tokenizer

Tokenizes the input from an edge into n-grams of given size(s).

Author: Otis Gospodnetic Adam Hiatt

Nested Class Summary
static class	EdgeNGramTokenizer.Side Specifies which side of the input the n-gram should be generated from

Field Summary
static int	DEFAULT_MAX_GRAM_SIZE
static int	DEFAULT_MIN_GRAM_SIZE
static EdgeNGramTokenizer.Side	DEFAULT_SIDE

Constructor Summary
	EdgeNGramTokenizer(Reader input, EdgeNGramTokenizer.Side side, int minGram, int maxGram) Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range
	EdgeNGramTokenizer(Reader input, String sideLabel, int minGram, int maxGram) Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Method Summary
Token	next() Returns the next token in the stream, or null at EOS.

Field Detail

DEFAULT_MAX_GRAM_SIZE

public static final int DEFAULT_MAX_GRAM_SIZE

DEFAULT_MIN_GRAM_SIZE

public static final int DEFAULT_MIN_GRAM_SIZE

DEFAULT_SIDE

public static final EdgeNGramTokenizer.Side DEFAULT_SIDE

Constructor Detail

EdgeNGramTokenizer

public EdgeNGramTokenizer(Reader input, EdgeNGramTokenizer.Side side, int minGram, int maxGram)

Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Parameters: input Reader holding the input to be tokenized side the Side from which to chop off an n-gram minGram the smallest n-gram to generate maxGram the largest n-gram to generate

EdgeNGramTokenizer

public EdgeNGramTokenizer(Reader input, String sideLabel, int minGram, int maxGram)

Creates EdgeNGramTokenizer that can generate n-grams in the sizes of the given range

Parameters: input Reader holding the input to be tokenized sideLabel the name of the Side from which to chop off an n-gram minGram the smallest n-gram to generate maxGram the largest n-gram to generate

Method Detail

public final Token next()

Returns the next token in the stream, or null at EOS.