LowerCaseTokenizer (Lucene 2.2.0 API)

Class LowerCaseTokenizer

public final class LowerCaseTokenizer extends LetterTokenizer

LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. It divides text at non-letters and converts them to lower case. While it is functionally equivalent to the combination of LetterTokenizer and LowerCaseFilter, there is a performance advantage to doing the two tasks at once, hence this (redundant) implementation.

Note: this does a decent job for most European languages, but does a terrible job for some Asian languages, where words are not separated by spaces.

Constructor Summary
	LowerCaseTokenizer(Reader in) Construct a new LowerCaseTokenizer.

Constructor Summary

LowerCaseTokenizer(Reader in)

Construct a new LowerCaseTokenizer.

Method Summary
protected char	normalize(char c) Collects only characters which satisfy Character#isLetter(char).

Method Summary

protected char

normalize(char c)

Collects only characters which satisfy Character#isLetter(char).

Method Detail

protected char normalize(char c)

Collects only characters which satisfy Character#isLetter(char).

Class LowerCaseTokenizer

Constructor Detail

LowerCaseTokenizer

Method Detail

normalize