org.apache.lucene.benchmark.byTask.feeds

Class BasicDocMaker

public abstract class BasicDocMaker extends Object implements DocMaker

Create documents for the test. Maintains counters of chars etc. so that sub-classes just need to provide textual content, and the create-by-size is handled here.

Config Params (default is in caps): doc.stored=true|FALSE
doc.tokenized=TRUE|false
doc.term.vector=true|FALSE
doc.store.body.bytes=true|FALSE //Store the body contents raw UTF-8 bytes as a field

Field Summary
protected Config	config
protected boolean	forever
protected Field.Index	indexVal
protected Field.Store	storeVal
protected Field.TermVector	termVecVal

Method Summary
protected void	addBytes(long n)
protected void	addUniqueBytes(long n)
protected void	collectFiles(File f, ArrayList inputFiles)
long	getByteCount()
int	getCount()
HTMLParser	getHtmlParser()
protected abstract DocData	getNextDocData() Return the data of the next document.
Document	makeDocument()
Document	makeDocument(int size)
long	numUniqueBytes()
void	printDocStatistics()
void	resetInputs()
void	setConfig(Config config)
void	setHTMLParser(HTMLParser htmlParser)

Field Detail

Method Detail

collectFiles

protected void collectFiles(File f, ArrayList inputFiles)

getNextDocData

protected abstract DocData getNextDocData()

Return the data of the next document. All current implementations can create docs forever. When the input data is exhausted, input files are iterated. This re-iteration can be avoided by setting doc.maker.forever to false (default is true).

Returns: data of the next document.

Throws: if cannot create the next doc data NoMoreDataException if data is exhausted (and 'forever' set to false).