public class SolrConnector extends BaseOutputConnector
| Modifier and Type | Class and Description |
|---|---|
protected class |
SolrConnector.SpecPacker
This class handles Solr connector version string packing/unpacking/interpretation.
|
| Modifier and Type | Field and Description |
|---|---|
static String |
_rcsid |
protected String |
allowAttributeName
The allow attribute name
|
protected String |
collectionName
Collection name (non-empty only if SolrCloud)
|
protected String |
contentAttributeName |
protected String |
createdDateAttributeName |
protected String |
denyAttributeName
The deny attribute name
|
protected boolean |
doCommits
Whether or not to commit
|
protected Map<String,String> |
excludedMimeTypes
Excluded mime types
|
protected String |
excludedMimeTypesString
Excluded mime types string
|
protected static long |
EXPIRATION_INTERVAL
Idle connection expiration interval
|
protected long |
expirationTime
Expiration
|
protected String |
fileNameAttributeName |
protected String |
idAttributeName |
protected Map<String,String> |
includedMimeTypes
Included mime types
|
protected String |
includedMimeTypesString
Included mime types string
|
protected String |
indexedDateAttributeName |
static String |
INGEST_ACTIVITY
Ingestion activity
|
protected Long |
maxDocumentLength
The maximum document length
|
protected String |
mimeTypeAttributeName |
protected String |
modifiedDateAttributeName |
protected String |
originalSizeAttributeName |
protected HttpPoster |
poster
Local connection
|
static String |
REMOVE_ACTIVITY
Document removal activity
|
protected boolean |
useExtractUpdateHandler
Use extractiing update handler?
|
protected boolean |
useUrlEncoding
Use URL encoding for field names?
|
currentContext, paramsDOCUMENTSTATUS_ACCEPTED, DOCUMENTSTATUS_REJECTED| Constructor and Description |
|---|
SolrConnector()
Constructor.
|
| Modifier and Type | Method and Description |
|---|---|
int |
addOrReplaceDocumentWithException(String documentURI,
VersionContext pipelineDescription,
RepositoryDocument document,
String authorityNameString,
IOutputAddActivity activities)
Add (or replace) a document in the output data store using the connector.
|
String |
check()
Test the connection.
|
boolean |
checkLengthIndexable(VersionContext outputDescription,
long length,
IOutputCheckActivity activities)
Pre-determine whether a document's length is indexable by this connector.
|
boolean |
checkMimeTypeIndexable(VersionContext outputDescription,
String mimeType,
IOutputCheckActivity activities)
Detect if a mime type is indexable or not.
|
void |
connect(ConfigParams configParameters)
Connect.
|
void |
disconnect()
Close the connection.
|
String[] |
getActivitiesList()
Return the list of activities that this connector supports (i.e.
|
String |
getFormCheckJavascriptMethodName(int connectionSequenceNumber)
Obtain the name of the form check javascript method to call.
|
String |
getFormPresaveCheckJavascriptMethodName(int connectionSequenceNumber)
Obtain the name of the form presave check javascript method to call.
|
VersionContext |
getPipelineDescription(Specification spec)
Get an output version string, given an output specification.
|
protected void |
getSession()
Set up a session
|
boolean |
isConnected()
This method is called to assess whether to count this connector instance should
actually be counted as being connected.
|
void |
noteJobComplete(IOutputNotifyActivity activities)
Notify the connector of a completed job.
|
void |
outputConfigurationBody(IThreadContext threadContext,
IHTTPOutput out,
Locale locale,
ConfigParams parameters,
String tabName)
Output the configuration body section.
|
void |
outputConfigurationHeader(IThreadContext threadContext,
IHTTPOutput out,
Locale locale,
ConfigParams parameters,
List<String> tabsArray)
Output the configuration header section.
|
void |
outputSpecificationBody(IHTTPOutput out,
Locale locale,
Specification os,
int connectionSequenceNumber,
int actualSequenceNumber,
String tabName)
Output the specification body section.
|
void |
outputSpecificationHeader(IHTTPOutput out,
Locale locale,
Specification os,
int connectionSequenceNumber,
List<String> tabsArray)
Output the specification header section.
|
protected static Map<String,String> |
parseMimeTypes(String mimeTypes)
Parse a mime type field into individual mime types in a hash
|
void |
poll()
This method is periodically called for all connectors that are connected but not
in active use.
|
String |
processConfigurationPost(IThreadContext threadContext,
IPostParameters variableContext,
Locale locale,
ConfigParams parameters)
Process a configuration post.
|
String |
processSpecificationPost(IPostParameters variableContext,
Locale locale,
Specification os,
int connectionSequenceNumber)
Process a specification post.
|
void |
removeDocument(String documentURI,
String outputDescription,
IOutputRemoveActivity activities)
Remove a document using the connector.
|
void |
viewConfiguration(IThreadContext threadContext,
IHTTPOutput out,
Locale locale,
ConfigParams parameters)
View configuration.
|
void |
viewSpecification(IHTTPOutput out,
Locale locale,
Specification os,
int connectionSequenceNumber)
View specification.
|
addOrReplaceDocument, checkDateIndexable, checkDocumentIndexable, checkDocumentIndexable, checkDocumentIndexable, checkLengthIndexable, checkMimeTypeIndexable, checkMimeTypeIndexable, checkURLIndexable, checkURLIndexable, getOutputDescription, noteAllRecordsRemoved, outputSpecificationBody, outputSpecificationBody, outputSpecificationHeader, outputSpecificationHeader, outputSpecificationHeader, processSpecificationPost, processSpecificationPost, requestInfo, viewSpecification, viewSpecificationclearThreadContext, deinstall, getConfiguration, install, outputConfigurationBody, outputConfigurationHeader, outputConfigurationHeader, pack, packFixedList, packList, packList, processConfigurationPost, setThreadContext, unpack, unpackFixedList, unpackList, viewConfigurationclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitclearThreadContext, deinstall, getConfiguration, install, setThreadContextpublic static final String _rcsid
public static final String INGEST_ACTIVITY
public static final String REMOVE_ACTIVITY
protected HttpPoster poster
protected long expirationTime
protected String allowAttributeName
protected String denyAttributeName
protected Long maxDocumentLength
protected String includedMimeTypesString
protected String excludedMimeTypesString
protected String idAttributeName
protected String originalSizeAttributeName
protected String modifiedDateAttributeName
protected String createdDateAttributeName
protected String indexedDateAttributeName
protected String fileNameAttributeName
protected String mimeTypeAttributeName
protected String contentAttributeName
protected boolean useExtractUpdateHandler
protected boolean useUrlEncoding
protected boolean doCommits
protected String collectionName
protected static final long EXPIRATION_INTERVAL
public String[] getActivitiesList()
getActivitiesList in interface IOutputConnectorgetActivitiesList in class BaseOutputConnectorpublic void connect(ConfigParams configParameters)
connect in interface IConnectorconnect in class BaseConnectorconfigParameters - is the set of configuration parameters, which
in this case describe the target appliance, basic auth configuration, etc. (This formerly came
out of the ini file.)public void poll()
throws ManifoldCFException
poll in interface IConnectorpoll in class BaseConnectorManifoldCFExceptionpublic boolean isConnected()
isConnected in interface IConnectorisConnected in class BaseConnectorpublic void disconnect()
throws ManifoldCFException
disconnect in interface IConnectordisconnect in class BaseConnectorManifoldCFExceptionprotected void getSession()
throws ManifoldCFException
ManifoldCFExceptionprotected static Map<String,String> parseMimeTypes(String mimeTypes) throws ManifoldCFException
ManifoldCFExceptionpublic String check() throws ManifoldCFException
check in interface IConnectorcheck in class BaseConnectorManifoldCFExceptionpublic VersionContext getPipelineDescription(Specification spec) throws ManifoldCFException, ServiceInterruption
getPipelineDescription in interface IPipelineConnectorgetPipelineDescription in class BaseOutputConnectorspec - is the current output specification for the job that is doing the crawling.ManifoldCFExceptionServiceInterruptionpublic boolean checkMimeTypeIndexable(VersionContext outputDescription, String mimeType, IOutputCheckActivity activities) throws ManifoldCFException, ServiceInterruption
checkMimeTypeIndexable in interface IPipelineConnectorcheckMimeTypeIndexable in class BaseOutputConnectoroutputDescription - is the document's output version.mimeType - is the mime type of the document.ManifoldCFExceptionServiceInterruptionpublic boolean checkLengthIndexable(VersionContext outputDescription, long length, IOutputCheckActivity activities) throws ManifoldCFException, ServiceInterruption
checkLengthIndexable in interface IPipelineConnectorcheckLengthIndexable in class BaseOutputConnectoroutputDescription - is the document's output version.length - is the length of the document.ManifoldCFExceptionServiceInterruptionpublic int addOrReplaceDocumentWithException(String documentURI, VersionContext pipelineDescription, RepositoryDocument document, String authorityNameString, IOutputAddActivity activities) throws ManifoldCFException, ServiceInterruption, IOException
addOrReplaceDocumentWithException in interface IPipelineConnectoraddOrReplaceDocumentWithException in class BaseOutputConnectordocumentURI - is the URI of the document. The URI is presumed to be the unique identifier which the output data store will use to process
and serve the document. This URI is constructed by the repository connector which fetches the document, and is thus universal across all output connectors.pipelineDescription - includes the description string that was constructed for this document by the getOutputDescription() method.document - is the document data to be processed (handed to the output data store).authorityNameString - is the name of the authority responsible for authorizing any access tokens passed in with the repository document. May be null.activities - is the handle to an object that the implementer of a pipeline connector may use to perform operations, such as logging processing activity,
or sending a modified document to the next stage in the pipeline.IOException - only if there's a stream error reading the document data.ManifoldCFExceptionServiceInterruptionpublic void removeDocument(String documentURI, String outputDescription, IOutputRemoveActivity activities) throws ManifoldCFException, ServiceInterruption
removeDocument in interface IOutputConnectorremoveDocument in class BaseOutputConnectordocumentURI - is the URI of the document. The URI is presumed to be the unique identifier which the output data store will use to process
and serve the document. This URI is constructed by the repository connector which fetches the document, and is thus universal across all output connectors.outputDescription - is the last description string that was constructed for this document by the getOutputDescription() method above.activities - is the handle to an object that the implementer of an output connector may use to perform operations, such as logging processing activity.ManifoldCFExceptionServiceInterruptionpublic void noteJobComplete(IOutputNotifyActivity activities) throws ManifoldCFException, ServiceInterruption
noteJobComplete in interface IOutputConnectornoteJobComplete in class BaseOutputConnectoractivities - is the handle to an object that the implementer of an output connector may use to perform operations, such as logging processing activity.ManifoldCFExceptionServiceInterruptionpublic void outputConfigurationHeader(IThreadContext threadContext, IHTTPOutput out, Locale locale, ConfigParams parameters, List<String> tabsArray) throws ManifoldCFException, IOException
outputConfigurationHeader in interface IConnectoroutputConfigurationHeader in class BaseConnectorthreadContext - is the local thread context.out - is the output to which any HTML should be sent.parameters - are the configuration parameters, as they currently exist, for this connection being configured.tabsArray - is an array of tab names. Add to this array any tab names that are specific to the connector.ManifoldCFExceptionIOExceptionpublic void outputConfigurationBody(IThreadContext threadContext, IHTTPOutput out, Locale locale, ConfigParams parameters, String tabName) throws ManifoldCFException, IOException
outputConfigurationBody in interface IConnectoroutputConfigurationBody in class BaseConnectorthreadContext - is the local thread context.out - is the output to which any HTML should be sent.parameters - are the configuration parameters, as they currently exist, for this connection being configured.tabName - is the current tab name.ManifoldCFExceptionIOExceptionpublic String processConfigurationPost(IThreadContext threadContext, IPostParameters variableContext, Locale locale, ConfigParams parameters) throws ManifoldCFException
processConfigurationPost in interface IConnectorprocessConfigurationPost in class BaseConnectorthreadContext - is the local thread context.variableContext - is the set of variables available from the post, including binary file post information.parameters - are the configuration parameters, as they currently exist, for this connection being configured.ManifoldCFExceptionpublic void viewConfiguration(IThreadContext threadContext, IHTTPOutput out, Locale locale, ConfigParams parameters) throws ManifoldCFException, IOException
viewConfiguration in interface IConnectorviewConfiguration in class BaseConnectorthreadContext - is the local thread context.out - is the output to which any HTML should be sent.parameters - are the configuration parameters, as they currently exist, for this connection being configured.ManifoldCFExceptionIOExceptionpublic String getFormCheckJavascriptMethodName(int connectionSequenceNumber)
getFormCheckJavascriptMethodName in interface IPipelineConnectorgetFormCheckJavascriptMethodName in class BaseOutputConnectorconnectionSequenceNumber - is the unique number of this connection within the job.public String getFormPresaveCheckJavascriptMethodName(int connectionSequenceNumber)
getFormPresaveCheckJavascriptMethodName in interface IPipelineConnectorgetFormPresaveCheckJavascriptMethodName in class BaseOutputConnectorconnectionSequenceNumber - is the unique number of this connection within the job.public void outputSpecificationHeader(IHTTPOutput out, Locale locale, Specification os, int connectionSequenceNumber, List<String> tabsArray) throws ManifoldCFException, IOException
outputSpecificationHeader in interface IPipelineConnectoroutputSpecificationHeader in class BaseOutputConnectorout - is the output to which any HTML should be sent.locale - is the preferred local of the output.os - is the current pipeline specification for this connection.connectionSequenceNumber - is the unique number of this connection within the job.tabsArray - is an array of tab names. Add to this array any tab names that are specific to the connector.ManifoldCFExceptionIOExceptionpublic void outputSpecificationBody(IHTTPOutput out, Locale locale, Specification os, int connectionSequenceNumber, int actualSequenceNumber, String tabName) throws ManifoldCFException, IOException
outputSpecificationBody in interface IPipelineConnectoroutputSpecificationBody in class BaseOutputConnectorout - is the output to which any HTML should be sent.locale - is the preferred local of the output.os - is the current pipeline specification for this job.connectionSequenceNumber - is the unique number of this connection within the job.actualSequenceNumber - is the connection within the job that has currently been selected.tabName - is the current tab name.ManifoldCFExceptionIOExceptionpublic String processSpecificationPost(IPostParameters variableContext, Locale locale, Specification os, int connectionSequenceNumber) throws ManifoldCFException
processSpecificationPost in interface IPipelineConnectorprocessSpecificationPost in class BaseOutputConnectorvariableContext - contains the post data, including binary file-upload information.locale - is the preferred local of the output.os - is the current pipeline specification for this job.connectionSequenceNumber - is the unique number of this connection within the job.ManifoldCFExceptionpublic void viewSpecification(IHTTPOutput out, Locale locale, Specification os, int connectionSequenceNumber) throws ManifoldCFException, IOException
viewSpecification in interface IPipelineConnectorviewSpecification in class BaseOutputConnectorout - is the output to which any HTML should be sent.locale - is the preferred local of the output.connectionSequenceNumber - is the unique number of this connection within the job.os - is the current pipeline specification for this job.ManifoldCFExceptionIOException