The Remote Engine is a key piece for executing components with Talend Pipeline Designer. To customize it for testing purposes, a helper tool is available.
The following content and the tool it describes are not officially supported as a Talend Cloud feature and are provided "as it" without any guarantee. |
Remote Engine Customizer
The remote-engine-customizer
is a simple tool that reads the Remote Engine docker-compose.yml
file, extracts the connectors
image and rebuilds it adding configured connectors to the image.
By default, it replaces the original image with the newly built one and saves the old file in the .remote_engine_customizer/backup/
directory of the Remote Engine.
Using the tool
The customizer is provided as a docker container.
You can run the tool using the register-component-archive
command:
docker run tacokit/remote-engine-customizer:${version} \
register-component-archive [options]
You can get more details about the tool with the following command:
docker run tacokit/remote-engine-customizer:${version} help
You can find out all the available versions at hub.docker.com/r/tacokit/remote-engine-customizer/tags. |
The command accepts the following parameters:
NAME
register-component-archive
SYNOPSIS
register-component-archive [options]
OPTIONS
--base-image=<String>
Base connector image to enrich with components.
default: auto
--cache-dir=<String>
Where to cache the image layers - useful when the command is launched multiple
times.
default: ${remote.engine.dir}/.remote_engine_customizer/cache
--component-archive=<String[]>
Component Archive (.car) to register/add in the Remote Engine.
--docker-configuration-environment=<String>
Docker daemon environment variables (as properties) if custom.
default:
--docker-configuration-path=<String>
Docker daemon executable path if custom.
--from-image-type=<ImageType>
Type of source image, can be useful if you don't use the default version
generation.
default: AUTO. enum: AUTO, DOCKER, REGISTRY
--registry-configuration-baseUrl=<String>
Registry base URL when image type is REGISTRY.
--registry-configuration-password=<String>
Registry password when image type is REGISTRY.
--registry-configuration-username=<String>
Registry username when image type is REGISTRY.
--remote-engine-dir=<String>
Where the remote engine folder is, it should host a docker-compose.yml file.
--target-image=<String>
Name of the target image, if not set it will be generated.
default: auto
--to-image-type=<ImageType>
Type of target image, it is generally just a local DOCKER image.
default: DOCKER. enum: AUTO, DOCKER, REGISTRY
--no-update-original-docker-compose
Should the Remote Engine docker-compose.yml be updated once the new image is
built.
--work-dir=<String>
Where to create temporary files needed for the build of the new image.
default: ${java.io.tmpdir}/remote-engine-customizer
When running using the docker container, you must mount several files/folders. For example:
docker run \
-v /opt/talend/pipeline-remote-engine/:/opt/talend/pipeline-remote-engine/ \ (1)
-v /opt/talend/downloads_exchange/:/opt/talend/downloads_exchange/ \ (2)
-v /var/run/docker.sock:/var/run/docker.sock \ (3)
tacokit/remote-engine-customizer:${version} \
register-component-archive \
--remote-engine-dir=/opt/talend/pipeline-remote-engine/
--component-archive=/opt/talend/downloads_exchange/mailio-1.0.0.car
1 | Mount the Remote Engine folder to let the docker image read and modify it. |
2 | Mount the folder hosting your local component if you want the Docker image to be able to use it. |
3 | Mount the docker daemon to let the program create a new docker image on the host. |
Once you have built the new image, you must restart the Remote Engine.
You can verify that the new image is available by running docker run --entrypoint sh -it [newimage] before.
|