Package eu.europa.esig.dss.service.http.commons

Class CommonsDataLoader

java.lang.Object

eu.europa.esig.dss.service.http.commons.CommonsDataLoader

All Implemented Interfaces:

DataLoader, Serializable

Direct Known Subclasses:

OCSPDataLoader, TimestampDataLoader

public class CommonsDataLoader
extends Object
implements DataLoader

Implementation of DataLoader for any protocol.

HTTP and HTTPS: using HttpClient which is more flexible for HTTPS without having to add the certificate to the JVM TrustStore. It takes into account a proxy management through ProxyPreferenceManager. The authentication is also supported.

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface eu.europa.esig.dss.spi.client.http.DataLoader

DataLoader.DataAndUrl

Field Summary

Fields

Modifier and Type	Field	Description
protected String	contentType	The content type value

Constructor Summary

Constructors

Constructor	Description
CommonsDataLoader()	The default constructor for CommonsDataLoader.
CommonsDataLoader(String contentType)	The constructor for CommonsDataLoader with defined content-type.

Method Summary

Modifier and Type	Method	Description
CommonsDataLoader	addAuthentication(HostConnection hostConnection, UserCredentials userCredentials)	Adds authentication credentials to the existing authenticationMap
CommonsDataLoader	addAuthentication(String host, int port, String scheme, String login, char[] password)	Adds authentication credentials to the existing authenticationMap
protected void	closeQuietly(org.apache.hc.client5.http.classic.methods.HttpUriRequestBase httpRequest, org.apache.hc.client5.http.impl.classic.CloseableHttpClient client)	Closes all the parameters quietly
protected org.apache.hc.client5.http.protocol.HttpClientContext	configurePreemptiveAuthentication(org.apache.hc.client5.http.protocol.HttpClientContext localContext, org.apache.hc.core5.http.HttpHost httpHost)	This method is used to configure preemptive authentication process for HttpClientContext, when required
protected byte[]	execute(org.apache.hc.client5.http.impl.classic.CloseableHttpClient client, org.apache.hc.client5.http.classic.methods.HttpUriRequest httpRequest)	Processes httpRequest and returns the byte array representing the response's content
protected byte[]	fileGet(String urlString)	Gets the file content by its URL
protected byte[]	ftpGet(String urlString)	This method retrieves data using FTP protocol .
byte[]	get(String urlString)	Execute a HTTP GET operation.
DataLoader.DataAndUrl	get(List<String> urlStrings)	Execute a HTTP GET operation.
Map<HostConnection,UserCredentials>	getAuthenticationMap()	Returns the current instance of the authentication map
int	getConnectionKeepAlive()	Gets the connection keep alive timeout.
int	getConnectionsMaxPerRoute()	Gets the maximum connections number per route.
int	getConnectionsMaxTotal()	Gets the maximum connections number.
int	getConnectionTimeToLive()	Gets the finite connection time to live.
String	getContentType()	Gets the content type
protected org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider	getCredentialsProvider()	Builds and returns a BasicCredentialsProvider configured with authenticationMap
HostnameVerifier	getHostnameVerifier()	Gets the hostname verifier
protected org.apache.hc.client5.http.impl.classic.CloseableHttpClient	getHttpClient(String url)	Gets the HTTP client
protected org.apache.hc.client5.http.impl.classic.HttpClientBuilder	getHttpClientBuilder(String url)	Gets the HttpClientBuilder for the url
org.apache.hc.core5.http.io.HttpClientResponseHandler<byte[]>	getHttpClientResponseHandler()	Returns the HttpClientResponseHandler response handler
protected org.apache.hc.core5.http.protocol.HttpContext	getHttpContext(org.apache.hc.core5.http.HttpHost httpHost)	Gets the HttpContext
protected org.apache.hc.core5.http.HttpHost	getHttpHost(org.apache.hc.client5.http.classic.methods.HttpUriRequest httpRequest)	Gets the HttpHost
protected org.apache.hc.client5.http.classic.methods.HttpGet	getHttpRequest(String url)	Gets the HTTP request
ProxyConfig	getProxyConfig()	Gets the proxy configuration
protected KeyStore	getSSLKeyStore()	Gets the SSL KeyStore
protected KeyStore	getSSLTrustStore()	Gets the SSL Trusted KeyStore
String[]	getSupportedSSLCipherSuites()	Gets supported SSL Cipher Suites
String[]	getSupportedSSLProtocols()	Gets supported SSL protocols
int	getTimeoutConnection()	Gets the connection timeout.
int	getTimeoutConnectionRequest()	Gets the connection request timeout.
int	getTimeoutResponse()	Gets the server response timeout.
int	getTimeoutSocket()	Gets the socket timeout.
org.apache.hc.core5.ssl.TrustStrategy	getTrustStrategy()	Gets the TrustStrategy
protected byte[]	httpGet(String url)	This method retrieves data using HTTP or HTTPS protocol and 'get' method.
boolean	isRedirectsEnabled()	Gets if redirect is enabled.
boolean	isUseSystemProperties()	Gets if the default system network properties shall be used
protected byte[]	ldapGet(String urlString)	This method retrieves data using LDAP protocol. - CRL from given LDAP url, e.g. ldap://ldap.infonotary.com/dc=identity-ca,dc=infonotary,dc=com - ex URL from AIA ldap://xadessrv.plugtests.net/CN=LevelBCAOK,OU=Plugtests_2015-2016,O=ETSI,C=FR?
byte[]	post(String url, byte[] content)	Executes a HTTP POST operation
void	setAuthenticationMap(Map<HostConnection,UserCredentials> authenticationMap)	Sets the authentication map NOTE: this method overrides the current instance of authenticationMap
void	setConnectionKeepAlive(int connectionKeepAlive)	Sets the connection keep alive timeout in milliseconds.
void	setConnectionsMaxPerRoute(int connectionsMaxPerRoute)	Sets the maximum connections number per route.
void	setConnectionsMaxTotal(int connectionsMaxTotal)	Sets the maximum connections number.
void	setConnectionTimeToLive(int connectionTimeToLive)	Sets the finite connection total time to live (TTL) in milliseconds.
void	setContentType(String contentType)	This allows to set the content type.
void	setHostnameVerifier(HostnameVerifier hostnameVerifier)	Sets a custom HostnameVerifier
void	setHttpClientResponseHandler(org.apache.hc.core5.http.io.HttpClientResponseHandler<byte[]> httpClientResponseHandler)	Sets the HttpClientResponseHandler<byte[]> response handler performing a processing of an HTTP client response and returns a byte array in case of success.
void	setKeyStoreAsTrustMaterial(boolean loadKeyStoreAsTrustMaterial)	Sets if the KeyStore shall be considered as a trust material (used for SSL connection)
void	setPreemptiveAuthentication(boolean preemptiveAuthentication)	Sets whether the preemptive authentication should be used.
void	setProxyConfig(ProxyConfig proxyConfig)	Sets the proxy configuration
void	setRedirectsEnabled(boolean redirectsEnabled)	Sets if redirect should be enabled.
void	setRetryStrategy(org.apache.hc.client5.http.HttpRequestRetryStrategy retryStrategy)	Sets a custom retry strategy
void	setSslKeystore(DSSDocument sslKeyStore)	Sets the SSL KeyStore
void	setSslKeystorePassword(char[] sslKeystorePassword)	Sets the KeyStore password.
void	setSslKeystoreType(String sslKeystoreType)	Sets the SSL KeyStore type
void	setSslProtocol(String sslProtocol)	This method sets the SSL protocol to be used
void	setSslTruststore(DSSDocument sslTrustStore)	Sets the SSL trust store NOTE: different from KeyStore!
void	setSslTruststorePassword(char[] sslTruststorePassword)	Sets the password for SSL truststore
void	setSslTruststoreType(String sslTruststoreType)	Sets the SSL TrustStore type
void	setSupportedSSLCipherSuites(String[] supportedSSLCipherSuites)	Sets supported SSL Cipher Suites
void	setSupportedSSLProtocols(String[] supportedSSLProtocols)	Sets supported SSL protocols
void	setTimeoutConnection(int timeoutConnection)	Sets the connection timeout in milliseconds.
void	setTimeoutConnectionRequest(int timeoutConnectionRequest)	Sets the connection request in milliseconds.
void	setTimeoutResponse(int timeoutResponse)	Sets the server response timeout in milliseconds.
void	setTimeoutSocket(int timeoutSocket)	Sets the socket timeout in milliseconds.
void	setTrustStrategy(org.apache.hc.core5.ssl.TrustStrategy trustStrategy)	Sets the TrustStrategy
void	setUseSystemProperties(boolean useSystemProperties)	Sets if the default system network properties shall be used Default: FALSE (system properties are not used) NOTE: all other configured property may override the default behavior!

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

contentType

protected String contentType

The content type value

Constructor Details

CommonsDataLoader

public CommonsDataLoader()

The default constructor for CommonsDataLoader.

CommonsDataLoader

public CommonsDataLoader(String contentType)

The constructor for CommonsDataLoader with defined content-type.

Parameters:

contentType - The content type of each request

Method Details

getTimeoutConnection

public int getTimeoutConnection()

Gets the connection timeout.

Returns:

the value (millis)

setTimeoutConnection

public void setTimeoutConnection(int timeoutConnection)

Sets the connection timeout in milliseconds. A negative value is interpreted as undefined (use system default).

Parameters:

timeoutConnection - the value (millis)

getTimeoutConnectionRequest

public int getTimeoutConnectionRequest()

Gets the connection request timeout.

Returns:

the value (millis)

setTimeoutConnectionRequest

public void setTimeoutConnectionRequest(int timeoutConnectionRequest)

Sets the connection request in milliseconds. A negative value is interpreted as undefined (use system default).

Parameters:

timeoutConnectionRequest - the value (millis)

getTimeoutResponse

public int getTimeoutResponse()

Gets the server response timeout.

Returns:

the value (millis)

setTimeoutResponse

public void setTimeoutResponse(int timeoutResponse)

Sets the server response timeout in milliseconds. A negative value is interpreted as undefined (use system default).

Parameters:

timeoutResponse - the value (millis)

getTimeoutSocket

public int getTimeoutSocket()

Gets the socket timeout.

Returns:

the value (millis)

setTimeoutSocket

public void setTimeoutSocket(int timeoutSocket)

Sets the socket timeout in milliseconds. A negative value is interpreted as undefined (use system default).

Parameters:

timeoutSocket - the value (millis)

getConnectionKeepAlive

public int getConnectionKeepAlive()

Gets the connection keep alive timeout.

Returns:

the value (millis)

setConnectionKeepAlive

public void setConnectionKeepAlive(int connectionKeepAlive)

Sets the connection keep alive timeout in milliseconds.

Parameters:

connectionKeepAlive - the value (millis)

getConnectionsMaxTotal

public int getConnectionsMaxTotal()

Gets the maximum connections number.

Returns:

the value (millis)

setConnectionsMaxTotal

public void setConnectionsMaxTotal(int connectionsMaxTotal)

Sets the maximum connections number.

Parameters:

connectionsMaxTotal - maximum number of connections

getConnectionsMaxPerRoute

public int getConnectionsMaxPerRoute()

Gets the maximum connections number per route.

Returns:

maximum number of connections per one route

setConnectionsMaxPerRoute

public void setConnectionsMaxPerRoute(int connectionsMaxPerRoute)

Sets the maximum connections number per route.

Parameters:

connectionsMaxPerRoute - maximum number of connections per one route

getConnectionTimeToLive

public int getConnectionTimeToLive()

Gets the finite connection time to live.

Returns:

connection time to live (millis)

setConnectionTimeToLive

public void setConnectionTimeToLive(int connectionTimeToLive)

Sets the finite connection total time to live (TTL) in milliseconds.

Parameters:

connectionTimeToLive - the finite connection time to live (millis)

isRedirectsEnabled

public boolean isRedirectsEnabled()

Gets if redirect is enabled.

Returns:

true if http redirects are allowed

setRedirectsEnabled

public void setRedirectsEnabled(boolean redirectsEnabled)

Sets if redirect should be enabled.

Parameters:

redirectsEnabled - true if http redirects are allowed

isUseSystemProperties

public boolean isUseSystemProperties()

Gets if the default system network properties shall be used

Returns:

TRUE if the default system network properties shall be used, FALSE otherwise

setUseSystemProperties

public void setUseSystemProperties(boolean useSystemProperties)

Sets if the default system network properties shall be used Default: FALSE (system properties are not used) NOTE: all other configured property may override the default behavior!

Parameters:

useSystemProperties - if the default system network properties shall be used

getContentType

public String getContentType()

Gets the content type

Returns:

the contentType

setContentType

public void setContentType(String contentType)

Description copied from interface: DataLoader

This allows to set the content type. Example: Content-Type "application/ocsp-request"

Specified by:

setContentType in interface DataLoader

Parameters:

contentType - to set the Content-Type

getProxyConfig

public ProxyConfig getProxyConfig()

Gets the proxy configuration

Returns:

associated ProxyConfig

setProxyConfig

public void setProxyConfig(ProxyConfig proxyConfig)

Sets the proxy configuration

Parameters:

proxyConfig - the proxyConfig to set

setSslProtocol

public void setSslProtocol(String sslProtocol)

This method sets the SSL protocol to be used

Parameters:

sslProtocol - the ssl protocol to be used

setSslKeystore

public void setSslKeystore(DSSDocument sslKeyStore)

Sets the SSL KeyStore

Parameters:

sslKeyStore - DSSDocument

setKeyStoreAsTrustMaterial

public void setKeyStoreAsTrustMaterial(boolean loadKeyStoreAsTrustMaterial)

Sets if the KeyStore shall be considered as a trust material (used for SSL connection)

Parameters:

loadKeyStoreAsTrustMaterial - if the KeyStore shall be considered as a trust material

setSslKeystoreType

public void setSslKeystoreType(String sslKeystoreType)

Sets the SSL KeyStore type

Parameters:

sslKeystoreType - String

setSslKeystorePassword

public void setSslKeystorePassword(char[] sslKeystorePassword)

Sets the KeyStore password. Please note that the password shall be the same for the keystore and the extraction of a corresponding key.

Parameters:

sslKeystorePassword - char array representing the password

setSslTruststore

public void setSslTruststore(DSSDocument sslTrustStore)

Sets the SSL trust store NOTE: different from KeyStore!

Parameters:

sslTrustStore - DSSDocument

setSslTruststorePassword

public void setSslTruststorePassword(char[] sslTruststorePassword)

Sets the password for SSL truststore

Parameters:

sslTruststorePassword - char array representing a password string

setSslTruststoreType

public void setSslTruststoreType(String sslTruststoreType)

Sets the SSL TrustStore type

Parameters:

sslTruststoreType - String

getAuthenticationMap

public Map<HostConnection,UserCredentials> getAuthenticationMap()

Returns the current instance of the authentication map

Returns:

a map between HostConnection and UserCredentials

setAuthenticationMap

public void setAuthenticationMap(Map<HostConnection,UserCredentials> authenticationMap)

Sets the authentication map NOTE: this method overrides the current instance of authenticationMap

Parameters:

authenticationMap - a map between HostConnection and UserCredentials

addAuthentication

public CommonsDataLoader addAuthentication(HostConnection hostConnection,
 UserCredentials userCredentials)

Adds authentication credentials to the existing authenticationMap

Parameters:

hostConnection - host connection details

userCredentials - user login credentials

Returns:

this (for fluent addAuthentication)

setPreemptiveAuthentication

public void setPreemptiveAuthentication(boolean preemptiveAuthentication)

Sets whether the preemptive authentication should be used. When set to TRUE, the client sends authentication details (i.e. user credentials) within the initial request to the remote host, instead of sending the credentials only after a request from the host. Please note that the preemptive authentication should not be used over an insecure connection. Default : FALSE (preemptive authentication is not used)

Parameters:

preemptiveAuthentication - whether the preemptive authentication should be used

addAuthentication

public CommonsDataLoader addAuthentication(String host,
 int port,
 String scheme,
 String login,
 char[] password)

Adds authentication credentials to the existing authenticationMap

Parameters:

host - host

port - port

scheme - scheme

login - login

password - password

Returns:

this (for fluent addAuthentication)

setRetryStrategy

public void setRetryStrategy(org.apache.hc.client5.http.HttpRequestRetryStrategy retryStrategy)

Sets a custom retry strategy

Parameters:

retryStrategy - HttpRequestRetryStrategy

getSupportedSSLProtocols

public String[] getSupportedSSLProtocols()

Gets supported SSL protocols

Returns:

an array if Strings

setSupportedSSLProtocols

public void setSupportedSSLProtocols(String[] supportedSSLProtocols)

Sets supported SSL protocols

Parameters:

supportedSSLProtocols - an array if Strings

getSupportedSSLCipherSuites

public String[] getSupportedSSLCipherSuites()

Gets supported SSL Cipher Suites

Returns:

an array if Strings

setSupportedSSLCipherSuites

public void setSupportedSSLCipherSuites(String[] supportedSSLCipherSuites)

Sets supported SSL Cipher Suites

Parameters:

supportedSSLCipherSuites - an array if Strings

getHostnameVerifier

public HostnameVerifier getHostnameVerifier()

Gets the hostname verifier

Returns:

HostnameVerifier

setHostnameVerifier

public void setHostnameVerifier(HostnameVerifier hostnameVerifier)

Sets a custom HostnameVerifier

Parameters:

hostnameVerifier - HostnameVerifier

getTrustStrategy

public org.apache.hc.core5.ssl.TrustStrategy getTrustStrategy()

Gets the TrustStrategy

Returns:

TrustStrategy

setTrustStrategy

public void setTrustStrategy(org.apache.hc.core5.ssl.TrustStrategy trustStrategy)

Sets the TrustStrategy

Parameters:

trustStrategy - TrustStrategy

getHttpClientResponseHandler

public org.apache.hc.core5.http.io.HttpClientResponseHandler<byte[]> getHttpClientResponseHandler()

Returns the HttpClientResponseHandler response handler

Returns:

HttpClientResponseHandler

setHttpClientResponseHandler

public void setHttpClientResponseHandler(org.apache.hc.core5.http.io.HttpClientResponseHandler<byte[]> httpClientResponseHandler)

Sets the HttpClientResponseHandler<byte[]> response handler performing a processing of an HTTP client response and returns a byte array in case of success.

Parameters:

httpClientResponseHandler - HttpClientResponseHandler

get

public byte[] get(String urlString)

Description copied from interface: DataLoader

Execute a HTTP GET operation.

Specified by:

get in interface DataLoader

Parameters:

urlString - the url to access

Returns:

byte array of obtained data or null

get

public DataLoader.DataAndUrl get(List<String> urlStrings)

Description copied from interface: DataLoader

Execute a HTTP GET operation. This method is used when many URls are available to access the same resource. The operation stops after the first successful download.

Specified by:

get in interface DataLoader

Parameters:

urlStrings - List of Strings representing the URLs to be used in sequential way to obtain the data.

Returns:

DataAndUrl representing the array of obtained data and used url, or null

ldapGet

protected byte[] ldapGet(String urlString)

This method retrieves data using LDAP protocol. - CRL from given LDAP url, e.g. ldap://ldap.infonotary.com/dc=identity-ca,dc=infonotary,dc=com - ex URL from AIA ldap://xadessrv.plugtests.net/CN=LevelBCAOK,OU=Plugtests_2015-2016,O=ETSI,C=FR?cACertificate;binary

Parameters:

urlString - String

Returns:

byte array

ftpGet

protected byte[] ftpGet(String urlString)

This method retrieves data using FTP protocol .

Parameters:

urlString - String url to retrieve data from

Returns:

byte array

fileGet

protected byte[] fileGet(String urlString)

Gets the file content by its URL

Parameters:

urlString - String to the file

Returns:

byte array of the file content

httpGet

protected byte[] httpGet(String url)

This method retrieves data using HTTP or HTTPS protocol and 'get' method.

Parameters:

url - to access

Returns:

byte array of obtained data or null

post

public byte[] post(String url,
 byte[] content)

Description copied from interface: DataLoader

Executes a HTTP POST operation

Specified by:

post in interface DataLoader

Parameters:

url - to access

content - the content to post

Returns:

byte array of obtained data

execute

protected byte[] execute(org.apache.hc.client5.http.impl.classic.CloseableHttpClient client,
 org.apache.hc.client5.http.classic.methods.HttpUriRequest httpRequest)
                  throws IOException

Processes httpRequest and returns the byte array representing the response's content

Parameters:

client - CloseableHttpClient

httpRequest - HttpUriRequest

Returns:

byte array representing the response's content

Throws:

IOException - if an exception occurs

getHttpHost

protected org.apache.hc.core5.http.HttpHost getHttpHost(org.apache.hc.client5.http.classic.methods.HttpUriRequest httpRequest)

Gets the HttpHost

Parameters:

httpRequest - HttpUriRequest

Returns:

HttpHost

getHttpContext

protected org.apache.hc.core5.http.protocol.HttpContext getHttpContext(org.apache.hc.core5.http.HttpHost httpHost)

Gets the HttpContext

Parameters:

httpHost - HttpHost

Returns:

HttpContext

configurePreemptiveAuthentication

protected org.apache.hc.client5.http.protocol.HttpClientContext configurePreemptiveAuthentication(org.apache.hc.client5.http.protocol.HttpClientContext localContext,
 org.apache.hc.core5.http.HttpHost httpHost)

This method is used to configure preemptive authentication process for HttpClientContext, when required

Parameters:

localContext - HttpClientContext

httpHost - HttpHost

Returns:

HttpClientContext

closeQuietly

protected void closeQuietly(org.apache.hc.client5.http.classic.methods.HttpUriRequestBase httpRequest,
 org.apache.hc.client5.http.impl.classic.CloseableHttpClient client)

Closes all the parameters quietly

Parameters:

httpRequest - HttpUriRequestBase

client - CloseableHttpClient

getSSLKeyStore

protected KeyStore getSSLKeyStore()
                           throws IOException,
GeneralSecurityException

Gets the SSL KeyStore

Returns:

KeyStore

Throws:

IOException - if IOException occurs

GeneralSecurityException - if GeneralSecurityException occurs

getSSLTrustStore

protected KeyStore getSSLTrustStore()
                             throws IOException,
GeneralSecurityException

Gets the SSL Trusted KeyStore

Returns:

KeyStore

Throws:

IOException - if IOException occurs

GeneralSecurityException - if GeneralSecurityException occurs

getHttpRequest

protected org.apache.hc.client5.http.classic.methods.HttpGet getHttpRequest(String url)
                                                                     throws URISyntaxException

Gets the HTTP request

Parameters:

url - String request url

Returns:

HttpGet

Throws:

URISyntaxException - if an exception occurs

getHttpClientBuilder

protected org.apache.hc.client5.http.impl.classic.HttpClientBuilder getHttpClientBuilder(String url)

Gets the HttpClientBuilder for the url

Parameters:

url - String request url

Returns:

HttpClientBuilder

getHttpClient

protected org.apache.hc.client5.http.impl.classic.CloseableHttpClient getHttpClient(String url)

Gets the HTTP client

Parameters:

url - String request url

Returns:

CloseableHttpClient

getCredentialsProvider

protected org.apache.hc.client5.http.impl.auth.BasicCredentialsProvider getCredentialsProvider()

Builds and returns a BasicCredentialsProvider configured with authenticationMap

Returns:

BasicCredentialsProvider