Component server and HTTP API

A test environment is available on Heroku and can be browsed using Talend Component Kit Server instance on Restlet Studio.

HTTP API

The HTTP API intends to expose most Talend Component Kit features over HTTP. It is a standalone Java HTTP server.

The WebSocket protocol is activated for the endpoints. Endpoints then use /websocket/v1 as base instead of /api/v1. See WebSocket for more details.

Here is the API:

Component Server API

To make sure that the migration can be enabled, you need to set the version the component was created with in the execution configuration that you send to the server (component version is in component the detail endpoint). To do that, use tcomp::component::version key.

Deprecated endpoints

Endpoints that are intended to disappear will be deprecated. A X-Talend-Warning header will be returned with a message as value.

WebSocket transport

You can connect yo any endpoint by:

  1. Replacing /api with /websocket

  2. Appending /<http method> to the URL

  3. Formatting the request as:

SEND
destination: <endpoint after v1>
<headers>

<payload>^@

For example:

SEND
destination: /component/index
Accept: application/json

^@

The response is formatted as follows:

MESSAGE
status: <http status code>
<headers>

<payload>^@
All endpoints are logged at startup. You can then find them in the logs if you have a doubt about which one to use.

If you don’t want to create a pool of connections per endpoint/verb, you can use the bus endpoint: /websocket/v1/bus. This endpoint requires that you add the destinationMethod header to each request with the verb value (GET by default):

SEND
destination: /component/index
destinationMethod: GET
Accept: application/json

^@

HTTPS activation

Using the server ZIP (or Docker image), you can configure HTTPS by adding properties to MEECROWAVE_OPTS. Assuming that you have a certificate in /opt/certificates/component.p12 (don’t forget to add/mount it in the Docker image if you use it), you can activate it as follows:

# use -e for Docker
#
# this skips the http port binding and only binds https on the port 8443, and setups the correct certificate
export MEECROWAVE_OPTS="-Dskip-http=true -Dssl=true -Dhttps=8443 -Dkeystore-type=PKCS12 -Dkeystore-alias=talend -Dkeystore-password=talend -Dkeystore-file=/opt/certificates/component.p12"

Web forms and REST API

The component-form library provides a way to build a component REST API facade that is compatible with React form library.

for example:

@Path("tacokit-facade")
@ApplicationScoped
public class ComponentFacade {
    private static final String[] EMPTY_ARRAY = new String[0];

    @Inject
    private Client client;

    @Inject
    private ActionService actionService;

    @Inject
    private UiSpecService uiSpecService;

    @Inject // assuming it is available in your app, use any client you want
    private WebTarget target;

    @POST
    @Path("action")
    public void action(@Suspended final AsyncResponse response, @QueryParam("family") final String family,
            @QueryParam("type") final String type, @QueryParam("action") final String action,
            final Map<String, Object> params) {
        client.action(family, type, action, params).handle((r, e) -> {
            if (e != null) {
                onException(response, e);
            } else {
                response.resume(actionService.map(type, r));
            }
            return null;
        });
    }

    @GET
    @Path("index")
    public void getIndex(@Suspended final AsyncResponse response,
            @QueryParam("language") @DefaultValue("en") final String language) {
        target
                .path("component/index")
                .queryParam("language", language)
                .request(APPLICATION_JSON_TYPE)
                .rx()
                .get(ComponentIndices.class)
                .toCompletableFuture()
                .handle((index, e) -> {
            if (e != null) {
                onException(response, e);
            } else {
                index.getComponents().stream().flatMap(c -> c.getLinks().stream()).forEach(
                        link -> link.setPath(link.getPath().replaceFirst("/component/", "/application/").replace(
                                "/details?identifiers=", "/detail/")));
                response.resume(index);
            }
            return null;
        });
    }

    @GET
    @Path("detail/{id}")
    public void getDetail(@Suspended final AsyncResponse response,
            @QueryParam("language") @DefaultValue("en") final String language, @PathParam("id") final String id) {
        target
                .path("component/details")
                .queryParam("language", language)
                .queryParam("identifiers", id)
                .request(APPLICATION_JSON_TYPE)
                .rx()
                .get(ComponentDetailList.class)
                .toCompletableFuture()
                .thenCompose(result -> uiSpecService.convert(result.getDetails().iterator().next()))
                .handle((result, e) -> {
                    if (e != null) {
                        onException(response, e);
                    } else {
                        response.resume(result);
                    }
                    return null;
                });
    }

    private void onException(final AsyncResponse response, final Throwable e) {
        final UiActionResult payload;
        final int status;
        if (WebException.class.isInstance(e)) {
            final WebException we = WebException.class.cast(e);
            status = we.getStatus();
            payload = actionService.map(we);
        } else if (CompletionException.class.isInstance(e)) {
            final CompletionException actualException = CompletionException.class.cast(e);
            log.error(actualException.getMessage(), actualException);
            status = Response.Status.BAD_GATEWAY.getStatusCode();
            payload = actionService.map(new WebException(actualException, -1, emptyMap()));
        } else {
            log.error(e.getMessage(), e);
            status = Response.Status.BAD_GATEWAY.getStatusCode();
            payload = actionService.map(new WebException(e, -1, emptyMap()));
        }
        response.resume(new WebApplicationException(Response.status(status).entity(payload).build()));
    }
}
the Client can be created using ClientFactory.createDefault(System.getProperty("app.components.base", "http://localhost:8080/api/v1")) and the service can be a simple new UiSpecService<>(). The factory uses JAX-RS if the API is available (assuming a JSON-B provider is registered). Otherwise, it tries to use Spring.

The conversion from the component model (REST API) to the uiSpec model is done through UiSpecService. It is based on the object model which is mapped to a UI model. Having a flat model in the component REST API allows to customize layers easily.

You can completely control the available components, tune the rendering by switching the uiSchema, and add or remove parts of the form. You can also add custom actions and buttons for specific needs of the application.

The /migrate endpoint was not shown in the previous snippet but if you need it, add it as well.

Using the UiSpec model without the tooling

<dependency>
  <groupId>org.talend.sdk.component</groupId>
  <artifactId>component-form-model</artifactId>
  <version>${talend-component-kit.version}</version>
</dependency>

This Maven dependency provides the UISpec model classes. You can use the Ui API (with or without the builders) to create UiSpec representations.

For Example:

final Ui form1 = ui()
    .withJsonSchema(JsonSchema.jsonSchemaFrom(Form1.class).build()) (1)
    .withUiSchema(uiSchema() (2)
        .withKey("multiSelectTag")
        .withRestricted(false)
        .withTitle("Simple multiSelectTag")
        .withDescription("This data list accepts values that are not in the list of suggestions")
        .withWidget("multiSelectTag")
        .build())
    .withProperties(myFormInstance) (3)
    .build();

final String json = jsonb.toJson(form1); (4)
1 The JsonSchema is extracted from reflection on the Form1 class. Note that @JsonSchemaIgnore allows to ignore a field and @JsonSchemaProperty allows to rename a property.
2 A UiSchema is programmatically built using the builder API.
3 An instance of the form is passed to let the serializer extract its JSON model.
4 The Ui model, which can be used by UiSpec compatible front widgets, is serialized.

The model uses the JSON-B API to define the binding. Make sure to have an implementation in your classpath. To do that, add the following dependencies:

<dependency>
  <groupId>org.apache.geronimo.specs</groupId>
  <artifactId>geronimo-jsonb_1.0_spec</artifactId>
  <version>1.0</version>
</dependency>
<dependency>
  <groupId>org.apache.geronimo.specs</groupId>
  <artifactId>geronimo-json_1.1_spec</artifactId>
  <version>1.0</version>
</dependency>
<dependency>
  <groupId>org.apache.johnzon</groupId>
  <artifactId>johnzon-jsonb</artifactId>
  <version>${johnzon.version}</version> <!-- 1.1.5 for instance -->
</dependency>

JavaScript integration

component-kit.js is no more available (previous versions stay on NPM) since it got replaced by @talend/react-containers. The previous import can be replaced by import kit from '@talend/react-containers/lib/ComponentForm/kit';.

Default JavaScript integration goes through the Talend UI Forms library and its Containers wrapper.

Documentation is now available on the previous link.

Logging

The logging uses Log4j2. You can specify a custom configuration by using the -Dlog4j.configurationFile system property or by adding a log4j2.xml file to the classpath.

Here are some common configurations:

  • Console logging:

<?xml version="1.0"?>
<Configuration status="INFO">
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="[%d{HH:mm:ss.SSS}][%highlight{%-5level}][%15.15t][%30.30logger] %msg%n"/>
    </Console>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>

Output messages look like:

[16:59:58.198][INFO ][           main][oyote.http11.Http11NioProtocol] Initializing ProtocolHandler ["http-nio-34763"]
  • JSON logging:

<?xml version="1.0"?>
<Configuration status="INFO">
  <Properties>
    <!-- DO NOT PUT logSource there, it is useless and slow -->
    <Property name="jsonLayout">{"severity":"%level","logMessage":"%encode{%message}{JSON}","logTimestamp":"%d{ISO8601}{UTC}","eventUUID":"%uuid{RANDOM}","@version":"1","logger.name":"%encode{%logger}{JSON}","host.name":"${hostName}","threadName":"%encode{%thread}{JSON}","stackTrace":"%encode{%xThrowable{full}}{JSON}"}%n</Property>
  </Properties>
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="${jsonLayout}"/>
    </Console>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>

Output messages look like:

{"severity":"INFO","logMessage":"Initializing ProtocolHandler [\"http-nio-46421\"]","logTimestamp":"2017-11-20T16:04:01,763","eventUUID":"8b998e17-7045-461c-8acb-c43f21d995ff","@version":"1","logger.name":"org.apache.coyote.http11.Http11NioProtocol","host.name":"TLND-RMANNIBUCAU","threadName":"main","stackTrace":""}
  • Rolling file appender:

<?xml version="1.0"?>
<Configuration status="INFO">
  <Appenders>
    <RollingRandomAccessFile name="File" fileName="${LOG_PATH}/application.log" filePattern="${LOG_PATH}/application-%d{yyyy-MM-dd}.log">
      <PatternLayout pattern="[%d{HH:mm:ss.SSS}][%highlight{%-5level}][%15.15t][%30.30logger] %msg%n"/>
      <Policies>
        <SizeBasedTriggeringPolicy size="100 MB" />
        <TimeBasedTriggeringPolicy interval="1" modulate="true"/>
      </Policies>
    </RollingRandomAccessFile>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="File"/>
    </Root>
  </Loggers>
</Configuration>

More details are available in the RollingFileAppender documentation.

You can compose previous layout (message format) and appenders (where logs are written).

UiSpec Server

The UiSpec server is a companion application for the Component Server. It provides a client to the Component Server which serves UiSpec payload to integrate with the client JavaScript UiForm library.

Coordinates

<dependency>
  <groupId>org.talend.sdk.component</groupId>
  <artifactId>component-server-proxy</artifactId>
  <version>${server-proxy.version}</version>
</dependency>

Configuring the UiSpec server

the configuration is read from system properties, environment variables, …​. If you use playx-microprofile-config, you can also use typesafe configuration.
Key Description Default

talend.component.proxy.actions.proposable.cached

If true the proposable (suggestion lists only depending on the server state) will be cached, otherwise they will be requested for each form rendering.

true

talend.component.proxy.application.home

A home location for relative path resolution (optional).

${playx.application.home}

talend.component.proxy.client.executor.threads

For the client executor, the number of threads.

64

talend.component.proxy.client.providers

List of JAX-RS providers to register on the client, at least a JSON-B one should be here.

-

talend.component.proxy.client.timeouts.connect

The connect timeout for the communication with the server.base in ms.

60000

talend.component.proxy.client.timeouts.read

The read timeout for the communication with the server.base in ms.

600000

talend.component.proxy.jcache.active

Should the server use jcache to store catalog information and refresh it with some polling. If so the keys talend.component.proxy.jcache.caches.$cacheName.expiry.duration, talend.component.proxy.jcache.caches.$cacheName.management.active and talend.component.proxy.jcache.caches.$cacheName.statistics.active will be read to create a JCache MutableConfiguration. Also note that if all the cachesshare the same configuration you can ignore the $cacheName layer.

true

talend.component.proxy.jcache.cache-key.name

A header to use in the cache key (to represent a tenant or equivalent).

-

talend.component.proxy.jcache.provider

Caching provider implementation to use (only set it if ambiguous).

-

talend.component.proxy.jcache.refresh.period

Number of seconds used to check if the server must be refreshed.

60

talend.component.proxy.processing.headers

The headers to append to the request when contacting the server. Format is a properties one. You can put a hardcoded value or a placeholder (${key}).In this case it will be read from the request attributes and headers.

-

talend.component.proxy.processing.uiSpec.patch

An optional location (absolute or resolved from APP_HOME environment variable). It can take an optional query parameter force which specifies if the startup should fail if the file is not resolved. The resolution is done per configuration type (datastore, dataset, …​) but fallbacks on default type if the file is not found.

The values can be keys in the resource bundle org.talend.sdk.component.proxy.enrichment.i18n.Messages. Use that for display names, placeholders etc…​The content

component-uispec-metadata.%s.json?force=false

talend.component.proxy.server.base

The base to contact the remote server (NOTE: it is recommanded to put a load balancer if you have multiple instances.)

-

Adding custom entries to the forms

As shown in the table above, you can customize the forms by type. The format reuses Talend Component Kit HTTP API (properties model) and defines two main types of extensions:

  1. prependProperties: Lists all the models of properties added to the form before the actual underlying form.

  2. appendProperties: Lists all the models of properties added to the form after the actual underlying form.

If you don’t specify a name, the path is used to deduce the name automatically.

Always make sure to define a root object for these properties. Do not use dots in the path value. It is recommended to prefix it with a $ character.

Adding custom converters (selecting the widget or rendering)

When developing a org.talend.sdk.component.form.internal.converter.CustomPropertyConverter CDI, the proxy adds it to the UiSpecService service and uses it with a high priority to convert the server model to a UiSpec model.

To make it a CDI bean, add @Dependent to the class and if you use the Play integration, customize the bean array: playx.cdi.beans.customs += {className: org.talend.myapp.MyConverter}.

This allows to use a custom @Ui API and advanced modeling when specific to applications.
Converters are sorted respecting to the @Priority value. If the annotation is missing, the priority defaults to 0.

Client in Play

The client to use to connect to the Talend Component Kit server is the CXF client, using HttpClient HC (NIO) transport. When you use the Play module, it can be configured with its standard properties prefixed by talend.component.proxy..

You can find more information on CXF website.

Defining a dropdown with all root configurations

The special dynamic_values action builtin::roots can be used for a dropdown filled with all available root types.

Here is a sample patch file:

{
  "prependProperties": [
    {
      "path": "$datasetMetadata",
      "type": "OBJECT"
    },
    {
      "path": "$datasetMetadata.type",
      "displayName": "Types",
      "type": "ENUM",
      "metadata": {
        "action::dynamic_values": "builtin::roots"
      }
    }
  ]
}

Populate a dropdown from a remote call

In some cases you will want to call a remote service giving you a list of data. For that there is a specific http builtin trigger which will be caught on the proxy and redirected to the configured server to map it as a dynamic_values trigger.

Assume you want to call http:://foo.com/bar propagating the cookies to get the proposals for the entry bar, then you will configure it this way:

{
  "prependProperties": [
    {
      "path": "$datasetMetadata",
      "type": "OBJECT"
    },
    {
      "path": "$datasetMetadata.bar",
      "displayName": "Bar",
      "type": "STRING",
      "metadata": {
        "action::dynamic_values": "builtin::http::dynamic_values(url=http:://foo.com/bar,headers=cookie)"
      }
    }
  ]
}

The configuration of the call is passed through a comma separated list in parenthesis as seen in previous snippet. The configuration entries are the following ones:

Name Description Sample

url

The target server to call. Note that placeholder are supported and filled from request headers (placeholderProvider).

proposals.service.com

headers

Comma separated list of headers to propagate (using placeholderProvider).

cookie,Authorization

accept

The Accept header. Note that current we only support to parse JSON output and it defaults to application/json.

application/v1+json

object

Boolean (default to false) to set to true if the response payload is an object and not a list.

true

objectKey

If object is true, the key to extract to get the list to process.

items

id

Name of the field to extract from an item of the response list to map to the identifier of the proposal (default to id).

id

name

Name of the field to extract from an item of the response list to map to the label of the proposal (default to name).

label

Populate a dropdown with a list of references

In some forms you will want to populate a dropdown with a list of entities available in your database. For that, the proxy will rely on a SPI called Integration. If you don’t implement it, it uses a Guice integration if Guice is in the classpath or a native CDI integration otherwise.

The integration will then let the proxy lookup a ReferenceService implementation which will provide from a configuration type type and name (for instance datastore, MyDataBaseConfig) the list of references you want to propose to the end user.

Here is very simple implementation:

@ApplicationScoped
public class ReferenceServiceImpl implements ReferenceService {
    @Inject
    private Database db;

    @Override
    public CompletionStage<Values> findReferencesByTypeAndName(final String type, final String name) {
        return CompletableFuture.completedFuture(new Values(db.findByTypeAndName(type, name).stream()
            .map(it -> new Values.Item(it.getId(), it.getName()))
            .collect(toList())));
    }

    @Override
    public CompletionStage<Form> findPropertiesById(final String id) {
        final Entity entity = db.findById(id);
        return CompletableFuture.completedFuture(Form.builder().properties(entity.getProperties()).formId(entity.getFormId()).build());
    }
}

Generally this kind of completion will be activated under the hood by the proxy using this kind of metadata:

{
  "action::dynamic_values": "builtin::references(type=xxxx,name=yyyy)"
}
Going further: make this field hidden when empty

In some cases you will want to hide this field if the proposals are ⇐ 1, in this case you can use a condition:

{
  "prependProperties": [
    {
      "path": "$datasetMetadata",
      "type": "OBJECT"
    },
    {
      "path": "$datasetMetadata.bar",
      "displayName": "Bar",
      "type": "STRING",
      "metadata": {
        "action::dynamic_values": "builtin::http::dynamic_values(url=http:://foo.com/bar,headers=cookie)",
        "condition::if::target": ".",
        "condition::if::value": "0,1",
        "condition::if::negate": "true",
        "condition::if::evaluationStrategy": "LENGTH",
      }
    }
  ]
}

Reloading the form based on the selected root

The builtin::root::reloadFromId action, with the reloadForm type, allows to reload the whole form:

{
  "path": "$datasetMetadata.type",
  "displayName": "Types",
  "type": "STRING",
  "metadata": {
    "action::dynamic_values": "builtin::roots", (1)
    "action::reloadForm": "builtin::root::reloadFromId", (2)
    "action::reloadForm::parameters": "."
  }
}
1 Prepopulating the dropdown with the list of datastores.
2 On selection of a datastore, refreshing the form with the new parameters.

It is common to have a dropdown with the list of roots and to reload the form when one is selected.

The client side (javascript) is now fully encapsulated in ComponentForm from @talend/ui.

Reloading the form based on a selected entity reference

The builtin::root::reloadFromParentEntityId action, with the reloadFromParentEntityId type, allows to reload the whole form:

{
  "path": "$datasetMetadata.reference",
  "displayName": "Reference",
  "type": "STRING",
  "metadata": {
    "action::reloadFromParentEntityId": "builtin::root::reloadFromParentEntityId",
    "action::reloadFromParentEntityId::parameters": "."
  }
}
references are automatically initialized if you use PropertiesService#filterProperties.

HTTP API

Component UiSpec Server

Overview

These endpoints allow to obtain UiSpec representations of the component/configuration types properties.

Version information

Version : v1

Contact information

Contact : Talend
Contact Email : contact@talend.com

License information

License : Apache 2.0
License URL : www.apache.org/licenses/LICENSE-2.0.html
Terms of service : null

URI scheme

Host : host:port
BasePath : /componentproxy/api/v1
Schemes : HTTP, HTTPS

Tags
  • action

  • configuration

  • configurations

  • dataset

  • datastore

  • form

  • icon

  • persistence

  • ui spec

  • uispec

Paths
This endpoint execute an action required by a form.
POST /actions/execute
Description

configuration types has action that can be executed using this endpoint

Parameters
Type Name Schema

Query

action
optional

string

Query

family
optional

string

Query

language
optional

string

Query

type
optional

string

Responses
HTTP Code Description Schema

200

successful operation
Headers :
Talend-Component-Server-Error (boolean) : This header indicate the error origin. true indicate an error from the component server, false indicate that the error is from this proxy.

400

This response is returned when the action is null

404

This response is returned when no action is found

520

This response is returned when the action raise an unhandled error

Consumes
  • application/json

Produces
  • application/json

Tags
  • action

  • configurations

Return all the available root configuration (Data store like) from the component server
GET /configurations
Description

Every configuration has an icon. In the response an icon key is returned. this icon key can be one of the bundled icons or a custom one. The consumer of this endpoint will need to check if the icon key is in the icons bundle otherwise the icon need to be gathered using the familyId from this endpoint configurations/{id}/icon

Responses
HTTP Code Description Schema

200

successful operation
Headers :
Talend-Component-Server-Error (boolean) : This header indicate the error origin. true indicate an error from the component server, false indicate that the error is from this proxy.

Consumes
  • application/json

Produces
  • application/json

Tags
  • configurations

  • datastore

Return a form description ( Ui Spec ) without a specific configuration
GET /configurations/form/initial/{type}
Parameters
Type Name Schema

Path

type
required

string

Responses
HTTP Code Description Schema

200

successful operation
Headers :
Talend-Component-Server-Error (boolean) : This header indicate the error origin. true indicate an error from the component server, false indicate that the error is from this proxy.

Consumes
  • application/json

Produces
  • application/json

Tags
  • configurations

  • dataset

  • datastore

  • form

  • ui spec

Return a form description ( Ui Spec ) of a specific configuration
GET /configurations/form/{id}
Parameters
Type Name Schema

Path

id
required

string

Responses
HTTP Code Description Schema

200

successful operation
Headers :
Talend-Component-Server-Error (boolean) : This header indicate the error origin. true indicate an error from the component server, false indicate that the error is from this proxy.

Consumes
  • application/json

Produces
  • application/json

Tags
  • configurations

  • dataset

  • datastore

  • form

  • ui spec

Return the configuration icon file in png format
GET /configurations/icon/{id}
Parameters
Type Name Schema

Path

id
required

string

Responses
HTTP Code Description Schema

200

successful operation
Headers :
Talend-Component-Server-Error (boolean) : This header indicate the error origin. true indicate an error from the component server, false indicate that the error is from this proxy.

Consumes
  • application/json

Produces
  • application/json

  • application/octet-stream

Tags
  • icon

Update a configuration.
POST /configurations/persistence/edit/{id}
Parameters
Type Name Schema

Path

id
required

string

Responses
HTTP Code Description Schema

200

successful operation
Headers :
Talend-Component-Server-Error (boolean) : This header indicate the error origin. true indicate an error from the component server, false indicate that the error is from this proxy.

Consumes
  • application/json

Produces
  • application/json

Tags
  • configurations

  • dataset

  • datastore

  • form

  • persistence

  • ui spec

Saves a configuration based on a type. Concretely it is the same as /persistence/save/{formId} but the formId is contained into the payload itself and marked in the metadata as such.
POST /configurations/persistence/save-from-type/{type}
Parameters
Type Name Schema

Path

type
required

string

Responses
HTTP Code Description Schema

200

successful operation
Headers :
Talend-Component-Server-Error (boolean) : This header indicate the error origin. true indicate an error from the component server, false indicate that the error is from this proxy.

Consumes
  • application/json

Produces
  • application/json

Tags
  • configurations

  • dataset

  • datastore

  • form

  • persistence

  • ui spec

Saves a configuration based on a form identifier.
POST /configurations/persistence/save/{formId}
Parameters
Type Name Schema

Path

formId
required

string

Responses
HTTP Code Description Schema

200

successful operation
Headers :
Talend-Component-Server-Error (boolean) : This header indicate the error origin. true indicate an error from the component server, false indicate that the error is from this proxy.

Consumes
  • application/json

Produces
  • application/json

Tags
  • configurations

  • dataset

  • datastore

  • form

  • persistence

  • ui spec

Definitions
CompletionStage

Type : object

CompletionStageByte[]

Type : object

CompletionStageCollectionSimplePropertyDefinition

Type : object

CompletionStageMapStringObject

Type : object

CompletionStageMapStringString

Type : object

CompletionStageNodes

Type : object

Condition
Name Schema

path
optional

string

shouldBe
optional

boolean

strategy
optional

string

values
optional

< object > array

EntityRef
Name Description Schema

id
optional

The identifier of the entity related to current request. It is generally thecreated entity of updated one.

string

JsonSchema
Name Schema

defaultValue
optional

object

description
optional

string

enumValues
optional

< string > array

id
optional

string

items
optional

maxItems
optional

integer (int32)

maxLength
optional

integer (int32)

maximum
optional

number (double)

minItems
optional

integer (int32)

minLength
optional

integer (int32)

minimum
optional

number (double)

pattern
optional

string

properties
optional

< string, JsonSchema > map

ref
optional

string

required
optional

< string > array

schema
optional

string

title
optional

string

type
optional

string

uniqueItems
optional

boolean

NameValue
Name Schema

name
optional

string

value
optional

string

Node
Name Description Schema

children
optional

The list of configuration reusing this one as a reference (can be created "next").

< string > array

familyId
optional

The identifier of the family of this configuration.

string

familyLabel
optional

The display name of the family of this configuration.

string

icon
optional

The icon of this configuration. If you use an existing bundle (@talend/ui/icon), ensure it is present by default and if not do a request using the family on the related endpoint.

string

id
optional

The identifier of this configuration/node.

string

label
optional

The display name of this configuration.

string

name
optional

The technical name of this node (it is human readable but not i18n friendly), useful for debug purposes.

string

version
optional

The version of this configuration for the migration management.

integer (int32)

Nodes
Name Description Schema

nodes
optional

The list of nodes matching the request. The key is the node identifier.

< string, Node > map

Option
Name Schema

path
optional

string

type
optional

string

Parameter
Name Schema

key
optional

string

path
optional

string

ProxyErrorPayload
Name Description Schema

code
optional

The error code independently of the locale and not as precise as a message (not context aware).

string

message
optional

A human readable message to help understanding the error

string

Trigger
Name Schema

action
optional

string

family
optional

string

onEvent
optional

string

options
optional

< Option > array

parameters
optional

< Parameter > array

type
optional

string

Ui
Name Schema

jsonSchema
optional

properties
optional

object

uiSchema
optional

< UiSchema > array

UiNode
Name Description Schema

metadata
optional

The metadata associated to the node if needed by the UI.

ui
optional

The ui specification corresponding to the requested node. It is literally the form representing this configuration.

Ui

UiSchema
Name Schema

autoFocus
optional

boolean

conditions
optional

< Condition > array

description
optional

string

disabled
optional

boolean

itemWidget
optional

string

items
optional

< UiSchema > array

key
optional

string

options
optional

< string, string > map

placeholder
optional

string

readOnly
optional

boolean

required
optional

boolean

restricted
optional

boolean

title
optional

string

titleMap
optional

< NameValue > array

triggers
optional

< Trigger > array

type
optional

string

widget
optional

string

There are two ways to call the save endpoint. If you don’t want to pass the form identifier and prefer to use a generic endpoint that simply passes the type of configuration you are configuring, then you need to modify your enrichment configuration to ensure that the form identifier is present and to specify which form field it is.

To do that, add the proxyserver::formId Boolean to the metadata:

{
  "path": "$datasetMetadata.type",
  "displayName": "Types",
  "type": "STRING",
  "metadata": {
    // other metadata as seen previously
    "proxyserver::formId": "true"
  }
}
Only the first property with proxyserver::formId set to "true" is used. The path cannot contain any array.

Configuring the server module

The server module contains several configurations that you can set in:

  • Environment variables.

  • System properties.

  • A file located based on the --component-configuration CLI option.

the configuration is read from system properties, environment variables, …​.
Key Description Default

talend.component.server.component.coordinates

A comma separated list of gav to locate the components

-

talend.component.server.component.extend.dependencies

Should the component extensions add required dependencies.

true

talend.component.server.component.registry

A property file where the value is a gav of a component to register (complementary with coordinates)

-

talend.component.server.documentation.active

Should the /documentation endpoint be activated.

true

talend.component.server.execution.dataset.retriever.timeout

How long the read execution endpoint can last (max)

180

talend.component.server.execution.pool.size

The size of the execution pool for runtime endpoints.

64

talend.component.server.execution.pool.wait

How long the application waits during shutdown for the execution tasks to complete

PT10S

talend.component.server.jaxrs.exceptionhandler.defaultMessage

If set it will replace any message for exceptions. Set to false to use the actual exception message.

false

talend.component.server.maven.repository

The local maven repository used to locate components and their dependencies

-

talend.component.server.monitoring.brave.reporter.async

When using url or kafka reporter, you can configure the async reporter with properties passed to this configuration entry.Ex: messageTimeout=5000,closeTimeout=5000.

console

talend.component.server.monitoring.brave.reporter.type

The brave reporter to use to send the spans. Supported values are [log, noop]. When configuration is needed,you can use this syntax to configure the repoter if needed: <name>(config1=value1, config2=value2), for example: url(endpoint=http://brave.company.com.

In log mode, if environment variable TRACING_ON doesn’t exist or is set to false, noop will be selected, and is set to true, TRACING_KAFKA_URL, TRACING_KAFKA_TOPIC and TRACING_SAMPLING_RATE will configure kafka reporter..

noop

talend.component.server.monitoring.brave.sampling.action.rate

The accuracy rate of the sampling for action endpoints.

-1

talend.component.server.monitoring.brave.sampling.component.rate

The accuracy rate of the sampling for component endpoints.

-1

talend.component.server.monitoring.brave.sampling.configurationtype.rate

The accuracy rate of the sampling for environment endpoints.

-1

talend.component.server.monitoring.brave.sampling.documentation.rate

The accuracy rate of the sampling for documentation endpoint.

-1

talend.component.server.monitoring.brave.sampling.environment.rate

The accuracy rate of the sampling for environment endpoints.

-1

talend.component.server.monitoring.brave.sampling.execution.rate

The accuracy rate of the sampling for execution endpoints.

1

talend.component.server.monitoring.brave.sampling.rate

The accuracy rate of the sampling.

-1.

talend.component.server.monitoring.brave.service.name

The name used by the brave integration (zipkin)

component-server

talend.component.server.security.command.handler

How to validate a command/request. Accepted values: securityNoopHandler.

securityNoopHandler

talend.component.server.security.connection.handler

How to validate a connection. Accepted values: securityNoopHandler.

securityNoopHandler

Docker

The server image is deployed on Docker. Its version is suffixed with a timestamp to ensure images are not overriden and can break your usage. You can check the available version on Docker hub.

Run

You can run the docker image by executing this command :

$ sudo docker run -p 8080:8080 tacokit/component-starter

Configure

You can set the env variable MEECROWAVE_OPTS to customize the server, by default it is installed in /opt/talend/component-kit.

Maven repository

The maven repository is the default one of the machine, you can change it setting the system property talend_component_server_maven_repository=/path/to/your/m2.

Deploy components to the server

If you want to deploy some components you can configure which ones in MEECROWAVE_OPTS (see server doc online) and redirect your local m2:

$ docker run \
       -p 8080:8080 \
       -v ~/.m2:/root/.m2 \
       -e MEECROWAVE_OPTS="-Dtalend.component.server.component.coordinates=g:a:v,g2:a2:v2,..." \
       component-server

Logging

The component server docker image comes with two log4j2 profile default and kafka. The logging profile can be changed by setting the environment variable TALEND_COMPONENT_LOG4J2_PROFILE to kafka the default profile is active by default.

default profile

The default profile has file and console logging capabilities. The console logging is off by default and you can activate it by setting CONSOLE_LOG_LEVEL environment variable to DEBUG, INFO, WARN or any other log level supported by log4j2. In practise and during development you will want to see the logs without connecting to the server by activating console logging.

Run docker image with console logging

sudo docker run -p 8080:8080 \
	-e CONSOLE_LOG_LEVEL=INFO \
	component-server

Kafka profile

Kafka profile let you send log to Kafka servers. The log are formatted in json and follow the layout defined by Talend and described here github.com/Talend/daikon/tree/master/daikon-logging/logging-event-layout

This profile require two environment variables

  • LOG_KAFKA_TOPIC : Kafka topic.

  • LOG_KAFKA_URL : A list of host/port pairs to use for establishing the initial connection to the Kafka cluster. This list should be in the form url:port separated by ,

Run docker image with kafka profile

sudo docker run -p 8080:8080 \
    -e TALEND_COMPONENT_LOG4J2_PROFILE=kafka \
    -e LOG_KAFKA_URL=`log kafka url:port` \
    -e LOG_KAFKA_TOPIC=`log kafka topic` \
	-e TRACING_KAFKA_URL=`tracing kafka url:port` \
	-e TRACING_KAFKA_TOPIC=`tracing kafka topic` \
	tacokit/component-server

Note : LOG_KAFKA_TOPIC will receive the application and the access logs and TRACING_KAFKA_TOPIC will receive brave tracing logs.

Tracing (Brave Monitoring)

The component server use github.com/openzipkin/brave to monitor request.

The tracing can be activated by setting environment variable TRACING_ON to true.

You can choose the reporter type by setting talend_component_server_monitoring_brave_reporter_type environment variable to log (this is the default value in this docker image) or to noop which will deactivate the tracing. Other type of reporter may be added in the future.

The tracing rate is configurable by setting environment variable TRACING_SAMPLING_RATE. This is the default sample rate for all the endpoints and has a default value of 0.1

You can define more accurate rate for every component server endpoint using those environment variables :

Environment variable Endpoint

talend_component_server_monitoring_brave_sampling_environment_rate

/api/v1/environment

talend_component_server_monitoring_brave_sampling_configurationtype_rate

/api/v1/configurationtype

talend_component_server_monitoring_brave_sampling_component_rate

/api/v1/component

talend_component_server_monitoring_brave_sampling_documentation_rate

/api/v1/documentation

talend_component_server_monitoring_brave_sampling_action_rate

/api/v1/action

talend_component_server_monitoring_brave_sampling_execution_rate

/api/v1/execution

Run docker image with tracing on

sudo docker run -p 8080:8080 \
	-e TRACING_ON=true \
	-e TRACING_SAMPLING_RATE = 0.1 \
	tacokit/component-server

Build the image yourself

You can build component starter server in docker following those instructions :

docker build --build-arg ARTIFACT_ID=component-starter-server \
             --build-arg SERVER_VERSION=`component starter server version` \
             --tag tacokit/component-server .
this assumes the project is built before you run that command.
Scroll to top