Component Server Vault Proxy

Browse the API description using OpenAPI.

A Vault proxy dedicated to the component server allows to safely manage credential encryption when operating in a Cloud environment.

Vault proxy

The Vault Proxy only concerns data marked as @Credential in the component configuration.

This Vault proxy:

  • receives an encrypted payload containing sensitive data from the remote engine, via HTTP.

  • decrypts the data

  • caches in memory the decrypted value of the data for performance reasons.

  • sends unencrypted data to the component server using HTTPS. An SSL certificate can be automatically generated and secures the data.

Vault proxy configuration

the configuration is read from system properties, environment variables, …​.
talend.vault.cache.client.certificate.acceptAny

Default value: false. Should any certificate be accepted - only for dev purposes.

talend.vault.cache.client.executor.server.core

Default value: 64. Thread pool core size for Component Server client.

talend.vault.cache.client.executor.server.keepAlive

Default value: 60000. Thread keep alive (in ms) for Component Server client thread pool.

talend.vault.cache.client.executor.server.max

Default value: 256. Thread pool max size for Component Server client.

talend.vault.cache.client.executor.vault.core

Default value: 64. Thread pool core size for Vault client.

talend.vault.cache.client.executor.vault.keepAlive

Default value: 60000. Thread keep alive (in ms) for Vault client thread pool.

talend.vault.cache.client.executor.vault.max

Default value: 256. Thread pool max size for Vault client.

talend.vault.cache.client.providers

JAX-RS fully qualified name of the provides (message body readers/writers) for vault and component-server clients.

talend.vault.cache.client.server.authorization

The token to use to call component-server if any.

talend.vault.cache.client.server.certificate.keystore.location

Where the keystore to use to connect to Component Server is located.

talend.vault.cache.client.server.certificate.keystore.password

Default value: changeit. The keystore password for talend.vault.cache.client.server.certificate.keystore.location.

talend.vault.cache.client.server.certificate.keystore.type

The keystore type for talend.vault.cache.client.server.certificate.keystore.location.

talend.vault.cache.client.server.certificate.truststore.type

The truststore type for talend.vault.cache.client.server.certificate.keystore.location.

talend.vault.cache.client.server.hostname.accepted

Default value: localhost,127.0.0.1,0:0:0:0:0:0:0:1. Valid hostnames for the Component Server certificates (see java.net.ssl.HostnameVerifier).

talend.vault.cache.client.timeout.connect

Default value: 30000. HTTP connection timeout to vault server.

talend.vault.cache.client.timeout.read

Default value: 30000. HTTP read timeout to vault server.

talend.vault.cache.client.vault.certificate.keystore.location

Where the keystore to use to connect to vault is located.

talend.vault.cache.client.vault.certificate.keystore.password

Default value: changeit. The keystore password for talend.vault.cache.client.vault.certificate.keystore.location.

talend.vault.cache.client.vault.certificate.keystore.type

The keystore type for talend.vault.cache.client.vault.certificate.keystore.location.

talend.vault.cache.client.vault.certificate.truststore.type

The truststore type for talend.vault.cache.client.vault.certificate.keystore.location.

talend.vault.cache.client.vault.hostname.accepted

Default value: localhost,127.0.0.1,0:0:0:0:0:0:0:1. Valid hostnames for the Vault certificates (see java.net.ssl.HostnameVerifier).

talend.vault.cache.jcache.cache.expiry

Default value: 3600. JCache expiry for decrypted values (ms).

talend.vault.cache.jcache.cache.management

Default value: false. Should JCache MBeans be registered.

talend.vault.cache.jcache.cache.statistics

Default value: false. Should JCache statistics be enabled.

talend.vault.cache.jcache.manager.properties

Default value: `. JCache `CacheManager properties used to initialized the instance.

talend.vault.cache.jcache.manager.uri

Default value: geronimo://simple-jcache.properties. Configuration for JCache setup, default implementation is Geronimo Simple Cache.

talend.vault.cache.jcache.maxCacheSize

Default value: 100000. JCache max size per cache.

talend.vault.cache.jcache.refresh.period

Default value: 30000. How often (in ms) the Component Server should be checked to invalidate the caches on the component parameters (to identify credentials).

talend.vault.cache.security.allowedIps

Default value: localhost,127.0.0.1,0:0:0:0:0:0:0:1. The IP or hosts allowed to call that server on /api/* if no token is passed.

talend.vault.cache.security.hostname.sanitizer

Default value: none. Enable to sanitize the hostname before testing them. Default to none which is a noop. Supported values are docker (for <folder>_<service>_<number>.<folder>_<network> pattern) and weave (for <prefix>_dataset_<number>.<suffix> pattern).

talend.vault.cache.security.tokens

Default value: -. The tokens enabling a client to call this server without being in allowedIp whitelist.

talend.vault.cache.service.auth.cantDecipherStatusCode

Default value: 422. Status code sent when vault can’t decipher some values.

talend.vault.cache.service.auth.refreshDelayMargin

Default value: 600000. How often (in ms) to refresh the vault token.

talend.vault.cache.service.auth.refreshDelayOnFailure

Default value: 10000. How often (in ms) to refresh the vault token in case of an authentication failure.

talend.vault.cache.service.decipher.skip.regex

Default value: vault\:v[0-9]+\:.*. The regex to whitelist ciphered keys, others will be passthrough in the output without going to vault.

talend.vault.cache.talendComponentKit.url

Base URL to connect to Component Server

talend.vault.cache.vault.auth.endpoint

Default value: v1/auth/engines/login. The vault path to retrieve a token.

talend.vault.cache.vault.auth.roleId

Default value: -. The vault role identifier to use to log in (if token is not set). - means it is ignored.

talend.vault.cache.vault.auth.secretId

Default value: -. The vault secret identifier to use to log in (if token is not set). - means it is ignored.

talend.vault.cache.vault.auth.token

Default value: -. The vault token to use to log in (will make roleId and secretId ignored). - means it is ignored.

talend.vault.cache.vault.decrypt.endpoint

Default value: v1/tenants-keyrings/decrypt/{x-talend-tenant-id}. The vault path to decrypt values. You can use the variable {x-talend-tenant-id} to replace by x-talend-tenant-id header value.

talend.vault.cache.vault.url

Base URL to connect to Vault.

Configuration mechanism

The configuration uses Microprofile Config for most entries. It means it can be passed through system properties and environment variables (by replacing dots with underscores and making the keys uppercase).

To configure a Docker image rather than a standalone instance, Docker Config and secrets integration allows you to read the configuration from files. You can customize the configuration of these integrations through system properties.

Docker integration provides a secure: support to encrypt values and system properties, when required.

It is fully implemented using the Apache Geronimo Microprofile Config extensions.

Adding the Vault Proxy to your Docker Compose

The YAML below is the recommended configuration to enable the Vault Proxy and Component Server to communicate over HTTPS.

Docker Compose

version: '3.2'

services:
  component-server: (1)
    healthcheck:
      timeout: 3s
      interval: 3s
      retries: 3
      test: curl --fail http://localhost:8080/api/v1/environment
    image: tacokit/component-server:${COMPONENT_SERVER_IMAGE_VERSION:-1.1.6_20190208104207}
    environment:
    - _JAVA_OPTIONS=
        -Dtalend.component.server.filter.secured.tokens=vault-proxy (2)
        -Dtalend.component.server.ssl.active=true (3)
        -Dtalend.component.server.ssl.keystore.location=/opt/talend/configuration/https.p12 (3)
        -Dtalend.component.server.ssl.keystore.type=PKCS12 (3)
        -Dtalend.component.server.component.registry=/opt/talend/connectors/component-registry.properties (4)
        -Dtalend.component.server.maven.repository=/opt/talend/connectors (4)
    volumes:
    - connectors:/opt/talend/connectors:ro  (4)
    - vault-proxy-configuration:/opt/talend/configuration (3)


  component-server-vault-proxy: (5)
    healthcheck:
      timeout: 3s
      interval: 3s
      retries: 3
      test: curl --fail http://localhost:8080/api/v1/proxy/environment
    image: tacokit/component-server-vault-proxy:${COMPONENT_SERVER_VAULT_PROXY_IMAGE_VERSION:-1.1.6_20190208104221}
    environment:
    - _JAVA_OPTIONS=
        -Dtalend.vault.cache.client.server.certificate.keystore.location=/opt/talend/configuration/https.p12 (6)
        -Dtalend.vault.cache.client.server.certificate.keystore.type=PKCS12 (6)
        -Dtalend.vault.cache.client.server.hostname.accepted=component-server (6)
        -Dtalend.vault.cache.client.server.authorization=vault-proxy (7)
        -Dtalend.vault.cache.talendComponentKit.url=https://component-server:8080/api/v1 (7)
        -Dtalend.vault.cache.vault.url=http://vault:8200 (8)
        -Dtalend.vault.cache.vault.auth.roleId=myrole (8)
        -Dtalend.vault.cache.vault.decrypt.endpoint=v1/something/decrypt/00000001 (8)
        -Dtalend.vault.cache.security.allowedIps=${COMPONENT_SERVER_VAULT_PROXY_CLIENT_IP:-127.0.0.1} (9)
    ports:
    - 9090:8080/tcp
    links: (10)
    - "component-server:component-server"
    # - "vault:vault"
    volumes:
    - vault-proxy-configuration:/opt/talend/configuration:ro

  connectors: (11)
    image: registry.datapwn.com/talend/connectors:${CONNECTORS_IMAGE_VERSION:-1.0.0_master_20190208091312}
    environment:
    - CONNECTORS_SETUP_OPTS=setup --wait-for-end
    volumes:
    - connectors:/opt/talend/connectors:ro

volumes: (12)
  connectors:
  vault-proxy-configuration:
1 The standard Component Server entry.
2 Ensures only a client with a particular token can call the server. It is similar to a shared secret and only allows to call the server in "remote" mode, since only the local mode is enabled by default.
3 Activates and configures the auto generation of a X.509 certificate which is used for the HTTPS connector on the server.
4 Binds the components to deploy into the server.
5 Definition of the Vault proxy service which handles the mediation between Vault and the Component Server.
6 Since both servers are colocalized, the generated certificate is inherited from the Component Server, which allows to create the client that connects to it.
7 Configuration of the base URL to connect to the server - see <10>.
8 Configuration of the vault connection and security information.
9 Ensuring that connecting from $COMPONENT_SERVER_VAULT_PROXY_CLIENT_IP to vault proxy is possible. Any other IP will be rejected.
10 Linking both services so they can communicate. It allows not to expose the Component Server port outside of its own container (no ports mapping in the Component Server service definition). Note that if your vault is a service, you can link it here as well.
11 Common component image service definition.
12 Volumes used by the services. The connectors has not changed but vault-proxy-configuration was added for the automatic HTTPS configuration.

This setup enables the Vault Proxy and Component Server to communicate. You can now use the Vault Proxy as if it was the Component Server, by using localhost:9090 (or any other host matching your deployment) instead of the Component Server directly.

Linking the Vault Proxy to the Component Server through HTTPS

When the Vault Proxy is enabled, ensure you configure HTTPS on the Component Server using the following parameters:

talend.component.server.ssl.active

true or false. Indicates if the SSL protocol is enabled.

talend.component.server.ssl.password

Keystore password.

talend.component.server.ssl.keystore.location

Path to Keystore.

talend.component.server.ssl.keystore.alias

Private key/certificate alias.

talend.component.server.ssl.keystore.type

Keystore type.

talend.component.server.ssl.keystore.generation.force

true or false.

talend.component.server.ssl.keystore.generation.command

Specifies if a custom command is to be used to generate the certificate.

talend.component.server.ssl.keypair.algorithm

Encryption algorithm. RSA by default.

talend.component.server.ssl.certificate.dname

Distinguished name.

talend.component.server.ssl.keypair.size

Size of the key. 2048 by default.

talend.component.server.ssl.port

SSL port to use.

Adding extensions to the instance

You can mount a volume in /opt/talend/component-kit-vault-proxy/custom/ and the jars in that folder which will be deployed with the server. Since the server relies on CDI (Apache OpenWebBeans) you can use that technology to enrich it, including JAX-RS endpoints, interceptors etc…​or just libraries needing to be in the JVM.

Scroll to top