Class LineDocSource
java.lang.Object
org.apache.lucene.benchmark.byTask.feeds.ContentItemsSource
org.apache.lucene.benchmark.byTask.feeds.ContentSource
org.apache.lucene.benchmark.byTask.feeds.LineDocSource
- All Implemented Interfaces:
Closeable,AutoCloseable
A
The expected format of each line is (arguments are separated by <TAB>): title, date, body. If a line is read in a different format, a
Config properties:
ContentSource reading one line at a time as a Document from a single file. This saves IO cost (over
DirContentSource) of recursing through a directory and opening a new file for every document.The expected format of each line is (arguments are separated by <TAB>): title, date, body. If a line is read in a different format, a
RuntimeException will be thrown. In
general, you should use this content source for files that were created with WriteLineDocTask.Config properties:
- docs.file=<path to the file>
- content.source.encoding - default to UTF-8.
- line.parser - default to
LineDocSource.HeaderLineParserif a header line exists which differs fromWriteLineDocTask.DEFAULT_FIELDSand toLineDocSource.SimpleLineParserotherwise.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic classLineDocSource.LineParserwhich sets field names and order by the header - any header - of the lines file.static classReader of a single input line intoDocData.static classLineDocSource.LineParserwhich ignores the header passed to its constructor and assumes simply that field names and their order are the same as inWriteLineDocTask.DEFAULT_FIELDS -
Field Summary
Fields inherited from class org.apache.lucene.benchmark.byTask.feeds.ContentItemsSource
encoding, forever, logStep, verbose -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoidclose()Called when reading from this content source is no longer required.getNextDocData(DocData docData) Returns the nextDocDatafrom the content source.voidResets the input for this content source, so that the test would behave as if it was just started, input-wise.voidSets theConfigfor this content source.Methods inherited from class org.apache.lucene.benchmark.byTask.feeds.ContentItemsSource
addBytes, addItem, collectFiles, getBytesCount, getConfig, getItemsCount, getTotalBytesCount, getTotalItemsCount, printStatistics, shouldLog
-
Constructor Details
-
LineDocSource
public LineDocSource()
-
-
Method Details
-
close
Description copied from class:ContentItemsSourceCalled when reading from this content source is no longer required.- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable- Specified by:
closein classContentItemsSource- Throws:
IOException
-
getNextDocData
Description copied from class:ContentSourceReturns the nextDocDatafrom the content source. Implementations must account for multi-threading, as multiple threads can call this method simultaneously.- Specified by:
getNextDocDatain classContentSource- Throws:
NoMoreDataExceptionIOException
-
resetInputs
Description copied from class:ContentItemsSourceResets the input for this content source, so that the test would behave as if it was just started, input-wise.NOTE: the default implementation resets the number of bytes and items generated since the last reset, so it's important to call super.resetInputs in case you override this method.
- Overrides:
resetInputsin classContentItemsSource- Throws:
IOException
-
setConfig
Description copied from class:ContentItemsSourceSets theConfigfor this content source. If you override this method, you must call super.setConfig.- Overrides:
setConfigin classContentItemsSource
-