component-runtime-junit is a test library that allows you to validate simple logic based on the Talend Component Kit tooling. To import it, add the following dependency to your project: This dependency also provides mocked components that you can use with your own component to create tests. The mocked components are provided under the test family: emitter : a mock of an input component collector : a mock of an output component The collector is "per thread" by default. If you are executing a Beam (or concurrent) job, it will not work. To switch to a JVM wide storage, set the talend.component.junit.handler.state system property to static (default being thread). You can do it in a maven-surefire-plugin execution. You can define a standard JUnit test and use the SimpleComponentRule rule: The rule can also be defined as a @ClassRule to start it once per class and not per test as with @Rule. To go further, you can add the ServiceInjectionRule rule, which allows to inject all the component family services into the test class by marking test class fields with @Service: The JUnit 5 integration is very similar to JUnit 4, except that it uses the JUnit 5 extension mechanism. The entry point is the @WithComponents annotation that you add to your test class, and which takes the component package you want to test. You can use @Injected to inject an instance of ComponentsHandler - which exposes the same utilities than the JUnit 4 rule - in a test class field : If you use JUnit 5 for the first time, keep in mind that the imports changed and that you need to use org.junit.jupiter.api.Test instead of org.junit.Test. Some IDE versions and surefire versions can also require you to install either a plugin or a specific configuration. As for JUnit 4, you can go further by injecting test class fields marked with @Service, but there is no additional extension to specify in this case: Streaming components have the issue to not stop by design. The Job DSL exposes two properties to help with that issue: streaming.maxRecords: enables to request a maximum number of records streaming.maxDurationMs: enables to request a maximum duration for the execution of the input You can set them as properties on the job: Using the test://collector component as shown in the previous sample stores all records emitted by the chain (typically your source) in memory. You can then access them using theSimpleComponentRule.getCollectedData(type). Note that this method filters by type. If you don’t need any specific type, you can use Object.class. The input mocking is symmetric to the output. In this case, you provide the data you want to inject: The component configuration is a POJO (using @Option on fields) and the runtime configuration (ExecutionChainBuilder) uses a Map. To make the conversion easier, the JUnit integration provides a SimpleFactory.configurationByExample utility to get this map instance from a configuration instance. Example: The same factory provides a fluent DSL to create the configuration by calling configurationByExample without any parameter. The advantage is to be able to convert an object as a Map or as a query string in order to use it with the Job DSL: It handles the encoding of the URI to ensure it is correctly done. When writing tests for your components, you can force the maxBatchSize parameter value by setting it with the following syntax: $configuration.$maxBatchSize=10. The SimpleComponentRule also allows to test a mapper unitarily. You can get an instance from a configuration and execute this instance to collect the output. Example: As for a mapper, a processor is testable unitary. However, this case can be more complex in case of multiple inputs or outputs. Example: The rule allows you to instantiate a Processor from your code, and then to collect the output from the inputs you pass in. There are two convenient implementations of the input factory: MainInputFactory for processors using only the default input. JoinInputfactory with the withInput(branch, data) method for processors using multiple inputs. The first argument is the branch name and the second argument is the data used by the branch. If needed, you can also implement your own input representation using org.talend.sdk.component.junit.ControllableInputFactory. The following artifact allows you to test against a Spark cluster: The testing relies on a JUnit TestRule. It is recommended to use it as a @ClassRule, to make sure that a single instance of a Spark cluster is built. You can also use it as a simple @Rule, to create the Spark cluster instances per method instead of per test class. The @ClassRule takes the Spark and Scala versions to use as parameters. It then forks a master and N slaves. Finally, the submit* method allows you to send jobs either from the test classpath or from a shade if you run it as an integration test. For example: This testing methodology works with @Parameterized. You can submit several jobs with different arguments and even combine it with Beam TestPipeline if you make it transient. The integration of that Spark cluster logic with JUnit 5 is done using the @WithSpark marker for the extension. Optionally, it allows you to inject—through @SparkInject—the BaseSpark handler to access the Spark cluster meta information. For example, its host/port. Example: Currently, SparkClusterRule does not allow to know when a job execution is done, even by exposing and polling the web UI URL to check. The best solution at the moment is to make sure that the output of your job exists and contains the right value. awaitability or any equivalent library can help you to implement such logic: To wait until a file exists and check that its content (for example) is the expected one, you can use the following logic: The HTTP JUnit module allows you to mock REST API very simply. The module coordinates are: This module uses Apache Johnzon and Netty. If you have any conflict (in particular with Netty), you can add the shaded classifier to the dependency. This way, both dependencies are shaded, which avoids conflicts with your component. It supports both JUnit 4 and JUnit 5. The concept is the exact same one: the extension/rule is able to serve precomputed responses saved in the classpath. You can plug your own ResponseLocator to map a request to a response, but the default implementation - which should be sufficient in most cases - looks in talend/testing/HTTP/_.json. Note that you can also put it in talend/testing/HTTP/.json. JUnit 4 setup is done through two rules: JUnit4HTTPApi, which is starts the server. JUnit4HTTPApiPerMethodConfigurator, which configures the server per test and also handles the capture mode. If you don’t use the JUnit4HTTPApiPerMethodConfigurator, the capture feature is disabled and the per test mocking is not available. For tests using SSL-based services, you need to use activeSsl() on the JUnit4HTTPApi rule. You can access the client SSL socket factory through the API handler: Sometimes the query parameters are sensitive and you don’t want to store them when capturing. In such cases, you can drop them from the captured data (.json) and the mock implementation will be able to match the request ignoring the query parameters. JUnit 5 uses a JUnit 5 extension based on the HTTPApi annotation that you can add to your test class. You can inject the test handler - which has some utilities for advanced cases - through @HTTPApiInject: The injection is optional and the @HTTPApi annotation allows you to configure several test behaviors. For tests using SSL-based services, you need to use @HTTPApi(useSsl = true). You can access the client SSL socket factory through the API handler: The strength of this implementation is to run a small proxy server and to auto-configure the JVM: HTTP[s].proxyHost, HTTP[s].proxyPort, HTTPsURLConnection#defaultSSLSocketFactory and SSLContext#default are auto-configured to work out-of-the-box with the proxy. It allows you to keep the native and real URLs in your tests. For example, the following test is valid: If you execute this test, it fails with an HTTP 400 error because the proxy does not find the mocked response. You can create it manually, as described in component-runtime-HTTP-junit, but you can also set the talend.junit.HTTP.capture property to the folder storing the captures. It must be the root folder and not the folder where the JSON files are located (not prefixed by talend/testing/HTTP by default). In most cases, use src/test/resources. If new File("src/test/resources") resolves the valid folder when executing your test (Maven default), then you can just set the system property to true. Otherwise, you need to adjust accordingly the system property value. When set to false, the capture is enabled. Instead, captures are saved in a false/ directory. When the tests run with this system property, the testing framework creates the correct mock response files. After that, you can remove the system property. The tests will still pass, using google.com, even if you disconnect your machine from the Internet. If you set the talend.junit.HTTP.passthrough system property to true, the server acts as a proxy and executes each request to the actual server - similarly to the capturing mode. With its @ParameterizedTest, you can want to customize the name of the output file for JUnit 5 based captures/mocks. Concretely you want to ensure the replay of the same method with different data lead to different mock files. By default the framework will use the display name of the test to specialize it but it is not always very friendly. If you want some more advanced control over the name you can use @HTTPApiName("myCapture.json") on the test method. To parameterize the name using @HTTPApiName, you can use the placeholders ${class} and ${method} which represents the declaring class and method name, and ${displayName} which represents the method name. Here is an example to use the same capture file for all repeated test: And here, the same example but using different files for each repetition:

The HTTP API intends to expose most Talend Component Kit features over HTTP. It is a standalone Java HTTP server. The WebSocket protocol is activated for the endpoints. Endpoints then use /websocket/v1 as base instead of /api/v1. See WebSocket for more details. Browse the API description using interface. To make sure that the migration can be enabled, you need to set the version the component was created with in the execution configuration that you send to the server (component version is in component the detail endpoint). To do that, use tcomp::component::version key. Endpoints that are intended to disappear will be deprecated. A X-Talend-Warning header will be returned with a message as value. You can connect yo any endpoint by: Replacing /api with /websocket Appending /<HTTP method> to the URL Formatting the request as: For example: The response is formatted as follows: All endpoints are logged at startup. You can then find them in the logs if you have a doubt about which one to use. If you don’t want to create a pool of connections per endpoint/verb, you can use the bus endpoint: /websocket/v1/bus. This endpoint requires that you add the destinationMethod header to each request with the verb value (GET by default): the configuration is read from system properties, environment variables, …. Default value: 1000. Maximum items a cache can store, used for index endpoints. A comma separated list of gav to locate the components Default value: ${home}/documentations. A component translation repository. This is where you put your documentation translations. Their name must follow the pattern documentation_${container-id}_language.adoc where ${container-id} is the component jar name (without the extension and version, generally the artifactId). Default value: true. Should the component extensions add required dependencies. If you deploy some extension, where they can create their dependencies if needed. Default value: 180000. Timeout for extension initialization at startup, since it ensures the startup wait extensions are ready and loaded it allows to control the latency it implies. A property file (or multiple comma separated) where the value is a gav of a component to register(complementary with coordinates). Note that the path can end up with or .properties to take into account all properties in a folder. Default value: true. Should the /documentation endpoint be activated. Note that when called on localhost the doc is always available. Default value: true. Should the /api/v1/environment endpoint be activated. It shows some internal versions and git commit which are not always desirable over the wire. Default value: false. Should the components using a @GridLayout support tab translation. Studio does not suppot that feature yet so this is not enabled by default. Default value: icons/%s.svg,icons/svg/%s.svg,icons/%s_icon32.png,icons/png/%s_icon32.png. These patterns are used to find the icons in the classpath(s). Default value: false. If set it will replace any message for exceptions. Set to false to use the actual exception message. Default value: false. Should the lastUpdated timestamp value of /environment endpoint be updated with server start time. Default value: en*=en fr*=fr zh*=zh_CN ja*=ja de*=de. For caching reasons the goal is to reduce the locales to the minimum required numbers. For instance we avoid fr and fr_FR which would lead to the same entries but x2 in terms of memory. This mapping enables that by whitelisting allowed locales, default being en. If the key ends with it means all string starting with the prefix will match. For instance fr will match fr_FR but also fr_CA. The local maven repository used to locate components and their dependencies Default value: false. Should the plugins be un-deployed and re-deployed. Default value: 600. Interval in seconds between each check if plugins re-loading is enabled. Specify a file to check its timestamp on the filesystem. This file will take precedence of the default ones provided by the talend.component.server.component.registry property (used for timestamp method). Default value: timestamp. Re-deploy method on a timestamp or connectors version change. By default, the timestamp is checked on the file pointed by talend.component.server.component.registry or talend.component.server.plugins.reloading.marker variable, otherwise we inspect the content of the CONNECTORS_VERSION file. Accepted values: timestamp, anything else defaults to connectors. Default value: false. Should the all requests/responses be logged (debug purposes - only work when running with CXF). Default value: securityNoopHandler. How to validate a command/request. Accepted values: securityNoopHandler. Default value: securityNoopHandler. How to validate a connection. Accepted values: securityNoopHandler. A folder available for the server - don’t forget to mount it in docker if you are using the image - which accepts subfolders named as component plugin id (generally the artifactId or jar name without the version, ex: jdbc). Each family folder can contain: a user-configuration.properties file which will be merged with component configuration system (see services). This properties file enables the function userJar(xxxx) to replace the jar named xxxx by its virtual gav (groupId:artifactId:version), a list of jars which will be merged with component family classpath Default value: auto. Should the implicit artifacts be provisionned to a m2. If set to auto it tries to detect if there is a m2 to provision - recommended, if set to skip it is ignored, else it uses the value as a m2 path. The configuration uses Microprofile Config for most entries. It means it can be passed through system properties and environment variables (by replacing dots with underscores and making the keys uppercase). To configure a Docker image rather than a standalone instance, Docker Config and secrets integration allows you to read the configuration from files. You can customize the configuration of these integrations through system properties. Docker integration provides a secure: support to encrypt values and system properties, when required. It is fully implemented using the Apache Geronimo Microprofile Config extensions. Using the server ZIP (or Docker image), you can configure HTTPS by adding properties to _JAVA_OPTIONS. Assuming that you have a certificate in /opt/certificates/component.p12 (don’t forget to add/mount it in the Docker image if you use it), you can activate it as follows: You can define simple queries on the configuration types and components endpoints. These two endpoints support different parameters. Queries on the configurationtype/index endpoint supports the following parameters: type id name metadata of the first configuration property as parameters. Queries on the component/index endpoint supports the following parameters: plugin name id familyId metadata of the first configuration property as parameters. In both cases, you can combine several conditions using OR and AND operators. If you combine more than two conditions, note that they are evaluated in the order they are written. Each supported parameter in a condition can be "equal to" (=) or "not equal to" (!=) a defined value (case-sensitive). For example: In this example, the query gets components that have a dataset and belong to the jdbc-component plugin, or components that are named input. The component-form library provides a way to build a component REST API facade that is compatible with React form library. for example: the Client can be created using ClientFactory.createDefault(System.getProperty("app.components.base", "HTTP://localhost:8080/api/v1")) and the service can be a simple new UiSpecService<>(). The factory uses JAX-RS if the API is available (assuming a JSON-B provider is registered). Otherwise, it tries to use Spring. The conversion from the component model (REST API) to the uiSpec model is done through UiSpecService. It is based on the object model which is mapped to a UI model. Having a flat model in the component REST API allows to customize layers easily. You can completely control the available components, tune the rendering by switching the uiSchema, and add or remove parts of the form. You can also add custom actions and buttons for specific needs of the application. The /migrate endpoint was not shown in the previous snippet but if you need it, add it as well. This Maven dependency provides the UISpec model classes. You can use the Ui API (with or without the builders) to create UiSpec representations. For Example: The model uses the JSON-B API to define the binding. Make sure to have an implementation in your classpath. To do that, add the following dependencies: The following module enables you to define through annotations a uispec on your own models: this can’t be used in components and is only intended for web applications. org.talend.sdk.component.form.uispec.mapper.api.service.UiSpecMapper enables to create a Ui instance from a custom type annotated with org.talend.sdk.component.form.uispec.mapper.api.model.View and org.talend.sdk.component.form.uispec.mapper.api.model.View.Schema. UiSpecMapper returns a Supplier and not directly an Ui because the ui-schema is re-evaluated when `get()̀ is called. This enables to update the title maps for example. Here is an example: This API maps directly the UiSpec model (json schema and ui schema of Talend UIForm). The default implementation of the mapper is available at org.talend.sdk.component.form.uispec.mapper.impl.UiSpecMapperImpl. Here is an example: The getTitleMapProviders() method will generally lookup a set of TitleMapProvider instances in your IoC context. This API is used to fill the titleMap of the form when a reference identifier is set on the @Schema annotation. component-kit.js is no more available (previous versions stay on NPM) and is replaced by @talend/react-containers. The previous import can be replaced by import kit from '@talend/react-containers/lib/ComponentForm/kit';. Default JavaScript integration goes through the Talend UI Forms library and its Containers wrapper. Documentation is now available on the previous link. The logging uses Log4j2. You can specify a custom configuration by using the -Dlog4j.configurationFile system property or by adding a log4j2.xml file to the classpath. Here are some common configurations: Console logging: Output messages look like: JSON logging: Output messages look like: Rolling file appender: More details are available in the RollingFileAppender documentation. You can compose previous layout (message format) and appenders (where logs are written). The server image is deployed on Docker. Its version is suffixed with a timestamp to ensure images are not overridden and can break your usage. You can check the available version on Docker hub. You can run the docker image by executing this command : You can set the env variable _JAVA_OPTIONS to customize the server, by default it is installed in /opt/talend/component-kit. The maven repository is the default one of the machine, you can change it setting the system property talend.component.server.maven.repository=/path/to/your/m2. If you want to deploy some components you can configure which ones in _JAVA_OPTIONS (see server doc online) and redirect your local m2: The component server docker image comes with two log4j2 profiles: TEXT (default) and JSON. The logging profile can be changed by setting the environment variable LOGGING_LAYOUT to JSON. Note that Component Server adds to these default Talend profiles the KAFKA profile. With this profile, all logs are sent to Kafka. You can check the exact configuration in the component-runtime/images/component-server-image/src/main/resources folder. The console logging is on at INFO level by default. You can customize it by setting the CONSOLE_LOG_LEVEL environment variable to DEBUG, INFO, WARN or to any other log level supported by log4j2. Run docker image with console logging: The JSON profile does the following: Logs on the console using the CONSOLE_LOG_LEVEL configuration as the default profile. It uses the formatting shown below. If the TRACING_KAFKA_URL environment variable is set, it logs the opentracing data on the defined Kafka using the topic TRACING_KAFKA_TOPIC. This level can be customized by setting the KAFKA_LOG_LEVEL environment variable (INFO by default). Events are logged in the following format: This profile is very close to the JSON profile and also adds the LOG_KAFKA_TOPIC and LOG_KAFKA_URL configuration. The difference is that it logs the default logs on Kafka in addition to the tracing logs. The component server uses Geronimo OpenTracing to monitor request. The tracing can be activated by setting the TRACING_ON environment variable to true. The tracing rate is configurable by setting the TRACING_SAMPLING_RATE environment variable. It accepts 0 (none) and 1 (all, default) as values to ensure the consistency of the reporting. You can find all the details on the configuration in org.talend.sdk.component.server.configuration.OpenTracingConfigSource. Run docker image with tracing on: By default, Geronimo OpenTracing will log the spans in a Zipking format so you can use the Kafka profile as explained before to wire it over any OpenTracing backend. You can register component server images in Docker using these instructions in the corresponding image directory: Docker Compose allows you to deploy the server with components, by mounting the component volume into the server image. docker-compose.yml example: If you want to mount it from another image, you can use this compose configuration: To run one of the previous compose examples, you can use docker-compose -f docker-compose.yml up. Only use the configuration related to port 5005 (in ports and the -agentlib option in _JAVA_OPTIONS) to debug the server on port 5005. Don’t set it in production. You can mount a volume in /opt/talend/component-kit/custom/ and the jars in that folder which will be deployed with the server. Since the server relies on CDI (Apache OpenWebBeans) you can use that technology to enrich it, including JAX-RS endpoints, interceptors etc…or just libraries needing to be in the JVM.

The framework provides built-in services that you can inject by type in components and actions. Type Description org.talend.sdk.component.api.service.cache.LocalCache Provides a small abstraction to cache data that does not need to be recomputed very often. Commonly used by actions for UI interactions. org.talend.sdk.component.api.service.dependency.Resolver Allows to resolve a dependency from its Maven coordinates. It can either try to resolve a local file or (better) creates for you a preinitialized classloader. javax.json.bind.Jsonb A JSON-B instance. If your model is static and you don’t want to handle the serialization manually using JSON-P, you can inject that instance. javax.json.spi.JsonProvider A JSON-P instance. Prefer other JSON-P instances if you don’t exactly know why you use this one. javax.json.JsonBuilderFactory A JSON-P instance. It is recommended to use this one instead of a custom one to optimize memory usage and speed. javax.json.JsonWriterFactory A JSON-P instance. It is recommended to use this one instead of a custom one to optimize memory usage and speed. javax.json.JsonReaderFactory A JSON-P instance. It is recommended to use this one instead of a custom one to optimize memory usage and speed. javax.json.stream.JsonParserFactory A JSON-P instance. It is recommended to use this one instead of a custom one to optimize memory usage and speed. javax.json.stream.JsonGeneratorFactory A JSON-P instance. It is recommended to use this one instead of a custom one to optimize memory usage and speed. org.talend.sdk.component.api.service.dependency.Resolver Allows to resolve files from Maven coordinates (like dependencies.txt for component). Note that it assumes that the files are available in the component Maven repository. org.talend.sdk.component.api.service.injector.Injector Utility to inject services in fields marked with @Service. org.talend.sdk.component.api.service.factory.ObjectFactory Allows to instantiate an object from its class name and properties. org.talend.sdk.component.api.service.record.RecordBuilderFactory Allows to instantiate a record. org.talend.sdk.component.api.service.record.RecordPointerFactory Allows to instantiate a RecordPointer which enables to extract a data from a Record based on jsonpointer specification. org.talend.sdk.component.api.service.record.RecordService Some utilities to create records from another one. It is typically what is used when you want to add an entry in a record and passthrough the other ones. It also provides a nice RecordVisitor API for advanced cases. org.talend.sdk.component.api.service.configuration.LocalConfiguration Represents the local configuration that can be used during the design. It is not recommended to use it for the runtime because the local configuration is usually different and the instances are distinct. You can also use the local cache as an interceptor with @Cached Every interface that extends HTTPClient and that contains methods annotated with @Request Lets you define an HTTP client in a declarative manner using an annotated interface. See the Using HTTPClient for more details. All these injected services are serializable, which is important for big data environments. If you create the instances yourself, you cannot benefit from these features, nor from the memory optimization done by the runtime. Prefer reusing the framework instances over custom ones. The local configuration uses system properties and the environment (replacing dots per underscores) to look up the values. You can also put a TALEND-INF/local-configuration.properties file with default values. This allows to use the local_configuration: syntax in @Ui annotation. Here is an example to read the default value of a property from the configuration: Ensure your key is unique across all components to avoid global overrides on the JVM. In practice, it is strongly recommended to always use the family as a prefix. Also note that you can use @Configuration("prefix") to inject a mapping of the LocalConfiguration in a component. It uses the same rules as for any configuration object. If you prefer to inject you configuration in a service, ensure to wrap it in a Supplier to always have an up to date version. If you want to ignore the local-configuration.properties, you can set the system property: talend.component.configuration.${componentPluginId}.ignoreLocalConfiguration=true. Here a sample @Configuration model: Here is how to use it from a service: And finally, here is how to use it in a component: it is recommended to convert this configuration in a runtime model in components to avoid to transport more than desired during the job distribution. You can access the API reference in the Javadocs. The HTTPClient usage is described in this section by using the REST API example below. Assuming that it requires a basic authentication header: GET /api/records/{id} - POST /api/records JSON payload to be created: {"id":"some id", "data":"some data"} To create an HTTP client that is able to consume the REST API above, you need to define an interface that extends HTTPClient. The HTTPClient interface lets you set the base for the HTTP address that the client will hit. The base is the part of the address that needs to be added to the request path to hit the API. It is now possible, and recommended, to use @Base annotation. Every method annotated with @Request in the interface defines an HTTP request. Every request can have a @Codec parameter that allows to encode or decode the request/response payloads. You can ignore the encoding/decoding for String and Void payloads. The interface should extend HTTPClient. In the codec classes (that implement Encoder/Decoder), you can inject any of your service annotated with @Service or @Internationalized into the constructor. Internationalization services can be useful to have internationalized messages for errors handling. The interface can be injected into component classes or services to consume the defined API. By default, /+json are mapped to JSON-P and /+xml to JAX-B if the model has a @XmlRootElement annotation. For advanced cases, you can customize the Connection by directly using @UseConfigurer on the method. It calls your custom instance of Configurer. Note that you can use @ConfigurerOption in the method signature to pass some Configurer configurations. For example, if you have the following Configurer: You can then set it on a method to automatically add the basic header with this kind of API usage: The framework provides in the component-api an OAuth1.Configurer which can be used as an example of configurer implementation. It expects a single OAuth1.Configuration parameter to be passed to the request as a @ConfigurationOption. Here is a sample showing how it can be used: By default, the client loads in memory the payload. In case of big payloads, it can consume too much memory. For these cases, you can get the payload as an InputStream: You can use the Response wrapper, or not.

TCOMP-2327: Upgrade cxf to 3.4.10 TCOMP-2328: Upgrade woodstox-core to 6.4.0 TCOMP-2294: Upgrade batik to 1.16 TCOMP-2295: Upgrade tomcat to 9.0.68 TCOMP-2045: Pass and read meta information about columns. studio-integration TCOMP-2096: Support BigDecimal type in DI integration schema-record studio studio-integration TCOMP-2070: Upgrade TSBI to 2.9.18-20220104141654 build component-server component-server-vault-proxy tsbi TCOMP-2105: Upgrade Tomcat to 9.0.60 component-server maven-plugin starter TCOMP-2030: Upgrade Tomcat to 9.0.54 due to CVE-2021-42340 TCOMP-2053: Migration failing when using custom java code in configuration TCOMP-2054: Upgrade log4j2 to 2.16.0 due to CVE-2021-44228 TCOMP-2048: RowstructVisitor should respect case in member not java convention TCOMP-2047: RecordBuilder in RowstructVisitor keeps values TCOMP-2046: Rowstruct visitor recreates schema at each incoming row TCOMP-1963: Missing IMetaDataColumn fields in guess schema TCOMP-1987: Avro record : Array of Array of records issue TCOMP-1988: Unable to run component-runtime connectors in Studio with JDK 17 TCOMP-2005: Non defined columns appear in schema TCOMP-2006: Support empty values for Numbers case TCOMP-2010: Error on Documentation build on "less" usage TCOMP-2020: talend-component-kit-intellij-plugin module build fails using Bintray (decomissioned) TCOMP-1900: Create jenkins release process for component-runtime TCOMP-1997: Enable plugins reloading according criteria TCOMP-2000: Upgrade netty to 4.1.68.Final TCOMP-2001: Upgrade Beam to 2.32.0 TCOMP-2007: connectors as a json object in Environment TCOMP-2009: Upgrade dockerfile-maven-plugin to 1.4.13 TCOMP-2016: UiSchema can’t hold advanced titleMap for more advanded datalist widgets TCOMP-2007: connectors as a json object in Environment TCOMP-1957: Avro schema builder issue TCOMP-1994: WebSocketClient$ClientException when executing action in Studio TCOMP-1923: Record : add metadata TCOMP-1990: Update jsoup to 1.14.2 due to CVE-2021-37714 TCOMP-1991: Update groovy to 3.0.9 due to CVE-2021-36373 / CVE-2021-36374 TCOMP-1992: Update lombok to 1.18.20 TCOMP-1993: Update TSBI to 2.9.0-20210907155713 TCOMP-1995: Expose the connectors (global) version in the "Environment" response TCOMP-1996: BaseService must not define equals & hashcode TCOMP-1994: WebSocketClient$ClientException when executing action in Studio TCOMP-1904: Delegate Avro record in AvroRecord seems to be invalid TCOMP-1967: goal uispec generation failure TCOMP-1983: fix module inclusion in dependencies.txt when build is java9+ TCOMP-1981: Allow to filter artifacts in car file generation TCOMP-1982: Allow to include extra artifacts in car file generation TCOMP-1876: Make schemaImpl immutable TCOMP-1885: Service Serializable TCOMP-1906: Redefine equals on RecordImpl TCOMP-1955: Upgrade cxf to 3.4.4 due to CVE-2021-30468 TCOMP-1966: Upgrade Tomcat to 9.0.50 due to CVE-2021-33037 TCOMP-1968: Upgrade maven to 3.8.1 TCOMP-1969: Upgrade Beam to 2.31.0 TCOMP-1970: Upgrade jackson to 2.12.1 TCOMP-1971: Upgrade Junit to 5.8.0-M1 TCOMP-1972: Upgrade slf4j to 1.7.32 TCOMP-1973: Upgrade log4j to 2.14.1 TCOMP-1974: Upgrade commons-compress to 1.21 due to CVE-2021-36090 TCOMP-1975: Upgrade TSBI to 2.8.2-20210722144648 TCOMP-1976: Upgrade meecrowave to 1.2.11 TCOMP-1977: Upgrade OpenWebBeans to 2.0.23 TCOMP-1978: Upgrade tomcat to 9.0.44 TCOMP-1979: Upgrade xbean to 4.20 TCOMP-1980: Upgrade meecrowave to 1.2.12 TCOMP-1967: goal uispec generation failure TCOMP-1935: After Variables doesn’t support custom object types TCOMP-1941: Maven goal talend-component:web fails on startup TCOMP-1947: Implement a retry strategy on failure in vault-client TCOMP-1948: Raised exception in component-server(s) should be serialized in json TCOMP-1952: IllegalArgumentException when the HTTP response return duplicated header. TCOMP-1939: Upgrade TSBI to Talend 2.7.2-20210616074048 TCOMP-1940: Upgrade Beam to 2.30.0 TCOMP-1941: Maven goal talend-component:web fails on startup TCOMP-1939: Upgrade TSBI to Talend 2.7.2-20210616074048 TCOMP-1919: Sanitize must force encoding file TCOMP-1925: Incorrect mapping of the parameters after arrays TCOMP-1937: Classpath not fully parsed in TSBI images TCOMP-1917: Add DatasetDiscovery annotation TCOMP-1707: Upgrade Geronimo :: Simple JCache to 1.0.5 TCOMP-1850: component-server with vault feature TCOMP-1907: Service monitor implementation & cleaning of grafana dashboard TCOMP-1921: Upgrade TSBI to 2.7.0-20210527090437 TCOMP-1930: Remove jsoup 1.7.x transitive dependency due to CVE-2015-6748 TCOMP-1936: Extend properties in Schema to use JsonValue TCOMP-1938: Add the german locale in the locale mapping TCOMP-1938: Add the german locale in the locale mapping TCOMP-1937: Classpath not fully parsed in TSBI images TCOMP-1919: Sanitize must force encoding file TCOMP-1886: Errors on Schema.sanitizeConnectionName TCOMP-1905: component-runtime fails to build with Java 11 TCOMP-1893: Upgrade to Beam 2.29.0 and use Beam’s Spark 3 specific module TCOMP-705: Support After variables TCOMP-1898: Add method to Record.Builder TCOMP-1910: Upgrade commons-io to 2.8.0 due to CVE-2021-29425 TCOMP-1911: Upgrade cxf to 3.4.3 due to CVE-2021-22696 TCOMP-1912: Upgrade TSBI to 2.6.7-20210503202416 TCOMP-1938: Add the german locale in the locale mapping TCOMP-1937: Classpath not fully parsed in TSBI images TCOMP-1880: Engine Server returns binary data instead of json (aka does not respect the compressed header) TCOMP-1886: Errors on Schema.sanitizeConnectionName TCOMP-1815: Support of ComponentException in migration TCOMP-1873: Add method getEntry on TCK Record Schema class TCOMP-1892: Upgrade Spark to 3.0.1 TCOMP-1888: Remove/change validation of ComponentException TCOMP-1894: Uniformize docker images entrypoints TCOMP-1895: Enhance coercion in RecordConverters TCOMP-1896: Upgrade TSBI to 2.6.4-20210331133410 TCOMP-1806: Double values are rounded to 5 decimal places in studio TCOMP-1851: HTTPClient implementation class is a Service with State TCOMP-1864: JsonSchemaConverter and johnzon-jsonschema 1.2.9+ look incompatible TCOMP-1866: Invalid number coercion on primitive type TCOMP-1869: byte[] handling is incorrect in dynamic column TCOMP-1871: Dynamic metadata name is not sanitized TCOMP-1861: Add a 'props' property in the Schema TCOMP-1863: Upgrade batik-codec to 1.14 due to CVE-2020-11988 TCOMP-1865: Upgrade cxf to 3.4.2 TCOMP-1867: Upgrade Apache Beam to 2.28.0 TCOMP-1878: Upgrade TSBI to 2.6.3-20210304090015 TCOMP-1688: Rewrite JsonSchema required rules to reflect component’s validation rules TCOMP-1857: Pojo conversion don’t support nested Objects TCOMP-1841: Add a SPI that would allow to add metadata to components TCOMP-1847: Upgrade Apache Beam to 2.27.0 TCOMP-1848: Upgrade bouncycastle to 1.68 due to CVE 2020-28052 TCOMP-1849: Proxify metrics component-server’s endpoint TCOMP-1852: Upgrade netty to v4.1.58.Final and ensure default HTTP testing module is java 11 friendly over ssl TCOMP-1854: Upgrade netty to 4.1.59.Final due to CVE-2021-21290 TCOMP-1855: Upgrade johnzon to 1.2.10 TCOMP-1856: Upgrade tomcat to 9.0.43 TCOMP-1841: Add a SPI that would allow to add metadata to components TCOMP-1852: Upgrade netty to v4.1.58.Final and ensure default HTTP testing module is java 11 friendly over ssl TCOMP-1854: Upgrade netty to 4.1.59.Final due to CVE-2021-21290 TCOMP-1848: Upgrade bouncycastle to 1.68 due to CVE 2020-28052 TCOMP-1839: Tomcat websocket server fails to start after tomcat 9.0.40 and meecrowave 1.2.10 TCOMP-1836: Upgrade OpenWebBeans to 2.0.20 TCOMP-1837: Upgrade xbean to 4.18 TCOMP-1838: Upgrade cxf to 3.4.1 TCOMP-1840: Upgrade tomcat to 9.0.41 TCOMP-1842: Upgrade jgit to 5.10.0.202012080955-r TCOMP-1844: Upgrade johnzon to 1.2.9 TCOMP-1845: Upgrade guava to 30.1-jre due to CVE-2020-8908 TCOMP-1848: Upgrade bouncycastle to 1.68 due to CVE 2020-28052 TCOMP-1839: Tomcat websocket server fails to start after tomcat 9.0.40 and meecrowave 1.2.10 TCOMP-1836: Upgrade OpenWebBeans to 2.0.20 TCOMP-1837: Upgrade xbean to 4.18 TCOMP-1827: Upgrade lombok to 1.18.16 TCOMP-1828: Change project’s versioning scheme TCOMP-1829: Upgrade TSBI to 2.5.3-20201201131449 TCOMP-1830: Upgrade Apache Beam to 2.26.0 TCOMP-1832: Upgrade HTTPclient to 4.5.13 due to CVE-2020-13956 TCOMP-1833: Upgrade spark to 2.4.7 TCOMP-1834: Upgrade groovy to 3.0.7 due to CVE-2020-17521 TCOMP-1787: ComponentManager can’t be re-created after it’s been closed TCOMP-1788: Invalid properties validation TCOMP-1801: Can’t look for resources in the classpath on Windows TCOMP-1761: Support of complete schema definition TCOMP-1725: Upgrade Tomcat to 9.0.40 TCOMP-1792: Uniform error message on component validation TCOMP-1808: Upgrade log4j2 to 2.14.0 TCOMP-1809: Update CXF to 3.3.8 due to CVE-2020-13954 TCOMP-1812: Upgrade junit to 4.13.1 due to CVE-2020-15250 TCOMP-1813: Upgrade jupiter to 5.7.0 TCOMP-1816: Apache Maven Shared Utils: OS Command Injection in Talend/component-runtime (master) and Talend/cloud-components TCOMP-1817: Upgrade gmavenplus-plugin to 1.11.0 TCOMP-1722: REST - Last / in endpoint is removed TCOMP-1757: Studio - context not set when call a @suggestable service TCOMP-1772: Code widget doesn’t allow multiline text TCOMP-1726: Update logos and colors TCOMP-1771: Record builder optimization (with static schema) TCOMP-1773: Upgrade log4j2 to 2.13.3 TCOMP-1774: Upgrade johnzon to 1.2.8 TCOMP-1775: Upgrade commons-lang3 to 3.11 TCOMP-1776: Upgrade commons-codec to 1.15 TCOMP-1777: Upgrade jgit to 5.9.0.202009080501-r TCOMP-1778: Upgrade jib-core to 0.15.0 TCOMP-1779: Upgrade batik to 1.13 TCOMP-1780: Upgrade TSBI to 2.4.0-20200925092052 TCOMP-1781: Upgrade asciidoctorj to 2.4.1 TCOMP-1782: Upgrade rrd4j to 3.7 TCOMP-1783: Upgrade netty to 5.0.0.Alpha2 TCOMP-1784: Upgrade ziplock to 8.0.4 TCOMP-1785: Upgrade JRuby to 9.2.13.0 TCOMP-1786: Upgrade to Apache Beam 2.24.0 TCOMP-1804: Upgrade to Apache Beam 2.25.0 TCOMP-1805: Upgrade TSBI to 2.5.0-20201030171201 TCOMP-1770: Performance loss on Ouput components in Studio TCOMP-1750: Deadlock at TPD job startup using the Component SDK and using the Workday component TCOMP-1759: Guess schema mixes columns returned by tck service TCOMP-1752: Make component-runtime class loader find classes in RemoteEngine JobServer TCOMP-1764: Upgrade to Apache Beam 2.23.0 TCOMP-1719: Header responses for icon not propagated correctly from Component-server-vault-proxy TCOMP-1733: NPE in Studio metadata connection with activeif on different layouts TCOMP-1734: Studio froze when installing a patch with azure-dls-gen2-1.10.0-component.car TCOMP-1736: JobImpl retrieves more than streaming.maxRecords parameter TCOMP-1739: Use scala version defined on parent for Spark related components TCOMP-1695: Support List type in Studio TCOMP-1737: Allow to force installation of an already existing component with the car bundle TCOMP-1728: Enforce use of the defined error contract in connectors TCOMP-1731: Make connectors docker image TSBI compliant TCOMP-1738: Upgrade to Apache Beam 2.22.0 TCOMP-1742: Upgrade johnzon to 1.2.7 TCOMP-1727: WebSocketContainer not present in ServletContext TCOMP-1696: Definition of an error contract to handle expected errors TCOMP-1729: Upgrade to Apache Beam 2.21.0 TCOMP-1730: Upgrade johnzon to 1.2.6 TCOMP-1719: Header responses for icon not propagated correctly from Component-server-vault-proxy TCOMP-1649: Tomcat bump to 9.0.31 broke talend-component:web goal TCOMP-1676: Starter-toolkit mvn package throws error when running for the first time TCOMP-1677: Using other types than String in Studio’s context values causes compilation error TCOMP-1679: Combination of @Required and @Suggestable on a field creates strange behaviour TCOMP-1682: Remove key attribute in UISchema for containers TCOMP-1686: antora helper function relativize corrupts documentation TCOMP-1694: [MAVEN PLUGIN] validateSvg argument is ineffective TCOMP-1698: UiSpecService injects a wrong property for suggestions and dynamic_values TCOMP-1718: Duplicated code in RecordConverters TCOMP-1702: Improve columns name TCOMP-1655: Upgrade jib-core to 0.13.1 TCOMP-1656: Upgrade log4j2 to 2.13.1 TCOMP-1657: Upgrade maven to 3.6.3 TCOMP-1658: Upgrade groovy to 3.0.2 TCOMP-1659: Upgrade lombok to 1.18.12 TCOMP-1660: Upgrade commons-compress to 1.20 TCOMP-1661: Upgrade commons-codec to 1.14 TCOMP-1662: Upgrade guava to 28.2-jre TCOMP-1663: Upgrade ziplock to 8.0.1 TCOMP-1664: Upgrade asciidoctorj to 2.2.0 and its dependencies TCOMP-1665: Upgrade jackson to 2.10.3 TCOMP-1666: Upgrade batik-codec to 1.12 TCOMP-1667: Upgrade jgit to 5.6.1.202002131546-r TCOMP-1668: Upgrade junit to 4.13 TCOMP-1669: Upgrade bouncycastle to 1.64 TCOMP-1670: Upgrade spark-core_2.11 to 2.4.5 TCOMP-1671: Upgrade maven-shade-plugin to 3.2.2 TCOMP-1672: Upgrade HTTPclient to 4.5.12 TCOMP-1673: Upgrade component-runtime-testing dependencies TCOMP-1674: Upgrade tomitribe-crest to 0.14 TCOMP-1678: Upgrade jgit to 5.7.0.202003090808-r TCOMP-1685: Provide docker images based on TSBI TCOMP-1687: More explicit exception messsage on reflection for findField TCOMP-1690: Upgrade netty to 4.1.48.Final TCOMP-1692: Update CXF to 3.3.6 due to CVE-2020-1954 TCOMP-1697: Update BouncyCastle to 1.65 TCOMP-1703: Upgrade log4j-2 to 2.13.2 TCOMP-1705: Upgrade to Apache Beam 2.20.0 TCOMP-1706: Upgrade OpenWebBeans to 2.0.16 TCOMP-1708: Upgrade groovy to 3.0.3 TCOMP-1710: Upgrade johnzon to 1.2.5 TCOMP-1711: Upgrade guava to 29.0-jre TCOMP-1712: Upgrade commons-lang3 to 3.10 TCOMP-1713: Upgrade jackson to 2.11.0 TCOMP-1714: Upgrade junit to 5.7.0-M1 TCOMP-1716: Upgrade maven shade plugin to 3.2.3 and misc libs TCOMP-1639: component-server incorrect response set in request TCOMP-1640: Ensure Intellij plugin works with Intellij Idea IU-201 TCOMP-1641: Upgrade OpenWebBeans to 2.0.15 TCOMP-1642: Upgrade Groovy to 3.0.1 TCOMP-1643: Add automatic scheduling eviction system on LocalCache TCOMP-1644: Upgrade log4j to 2.13.0 TCOMP-1645: Ensure correct wording is used in @Documentation TCOMP-1647: Upgrade netty to 4.1.45.Final TCOMP-1648: Unsafe Dependancy Resolution on jcommander TCOMP-1638: Inject services to delegate in proxy TCOMP-1619: Handle correctly DATETIME field type on AvroRecord TCOMP-1622: [DOC] @Icon is not supported on datastore/dataset TCOMP-1623: Change scheme for maven repos TCOMP-1628: Manage BigDecimal in RecordConverter TCOMP-1629: Ensure LocalConfiguration environment source replace dot with _ TCOMP-1630: Avoid NPE when configurationByExample() is called in a list of primitive without values TCOMP-1631: int attribute in pojo is transformed to double in a Record TCOMP-1632: Add a way to evict cached data from LocalCache TCOMP-1616: Upgrade OpenWebBeans to 2.0.14 in component-server and component-server-vault-proxy TCOMP-1617: Move mocked api results to github pages TCOMP-1618: Upgrade Junit to 5.6.0 TCOMP-1620: Upgrade to Apache Beam 2.18.0 TCOMP-1621: Upgrade to Johnzon 1.2.3 TCOMP-1624: @Service does not support list injections TCOMP-1625: Upgrade to xbean 4.16 TCOMP-1626: Ensure ContainerListenerExtensions can be sorted TCOMP-1627: Upgrade to Apache Beam 2.19.0 TCOMP-1633: Upgrade Groovy to 3.0.0 TCOMP-1634: Upgrade tomcat to 9.0.31 TCOMP-1596: Windows URI are broken TCOMP-1597: HTTPclient does not support multi query parameters TCOMP-1598: validator task uses ENGLISH locale to validate instead of root one TCOMP-1612: Starter toolkit shouldn’t use the default 'STAR' icon in demo component TCOMP-1585: Upgrade netty to 4.1.43.Final TCOMP-1586: Upgrade ziplock to v8.0.0 TCOMP-1587: Upgrade jib to v0.12.0 TCOMP-1588: Upgrade JRuby to v9.2.9.0 TCOMP-1589: Upgrade crest to v0.11.0 TCOMP-1591: Update to Tomcat 9.0.29 TCOMP-1592: Update to Johnzon 1.2.2 TCOMP-1593: Update to OpenWebBeans 2.0.13 TCOMP-1595: Infinite partitionmapper shouldn’t require assesor TCOMP-1599: More unsafe usage tolerance on JVM versions TCOMP-1600: Upgrade to Tomcat 9.0.30 TCOMP-1606: Ensure job dsl can stop infinite inputs TCOMP-1608: Upgrade geronimo openapi to 1.0.12 TCOMP-1609: Ensure Intellij plugin works with Intellij Idea 2019 TCOMP-1611: Upgrade to Apache Beam 2.17.0 TCOMP-1613: Upgrade cxf to 3.3.5 TCOMP-1614: Upgrade groovy to 3.0.0-rc3 TCOMP-1615: Upgrade OpenWebBeans to 2.0.14 TCOMP-1560: Min and Max error message during configuration validation are reversed TCOMP-1563: Web Tester does not work anymore (maven/gradle goal/task) TCOMP-1573: Body encoder is called twice for each query TCOMP-1582: Deploy to Nexus 3.15 caused "Provided url doesn’t respond neither to Nexus 2 nor to Nexus 3 endpoints" TCOMP-1576: Add the possibility to desactivate HTTP client redirection in HTTP Configurer TCOMP-1559: Support configuration of the maxBatchSize enablement TCOMP-1561: Custom action type shouldn’t need to be enforced to define a family method TCOMP-1562: Support JsonObject type in actions TCOMP-1564: Move to java.nio.Path instead of java.io.File in component-runtime-manager stack where possible TCOMP-1565: Upgade to Junit Jupiter 5.6.0-M1 TCOMP-1566: Don’t compute jvmMarkers per component module but once for all TCOMP-1567: Cache Artifact path in case of reuse TCOMP-1568: Lazily create the container services TCOMP-1569: Upgrade starter to gradle 6.0-rc1 TCOMP-1570: Ensure starter adds _placeholder entries in Messages.properties TCOMP-1571: Support [length] syntax to change array configuration TCOMP-1572: Validate that @Option is not used on final fields TCOMP-1574: Upgrade to CXF 3.3.4 TCOMP-1575: Upgrade to Spark 2.4.4 TCOMP-1577: Upgrade to xbean 4.15 TCOMP-1578: Upgrade asciidoctor-pdf to v1.5.0-beta.7 TCOMP-1581: Support JUnit5 meta annotations for our extensions TCOMP-1752: Make component-runtime class loader find classes in RemoteEngine JobServer TCOMP-1702: Improve columns name TCOMP-1685: Provide docker images based on TSBI TCOMP-1558: org.talend.sdk.component.api.service.record.RecordService must be serializable TCOMP-1548: Basic Remote Engine Customizer TCOMP-1550: Component configuration instantiation can be slow for complex configurations TCOMP-1551: ObjectFactory should default to fieldproperties when field injection is activated TCOMP-1553: Simplify and widden excluded classes for with transformer support TCOMP-1555: Upgrade to Tomcat 9.0.27 TCOMP-1556: Studio short, byte, BigDecimal and char types are wrong handled TCOMP-1557: Upgrade to Beam 2.16.0 TCOMP-1509: Intellij plugin does not declare java module preventing the plugin to run under last versions TCOMP-1526: Upgrade talend UI bundle (js) to 4.6.0 TCOMP-1533: JSON-B API does not enable to combine multiple adapters or (de)serializers in JsonbConfig TCOMP-1536: @DefaultValue ignored in documentation generation TCOMP-1541: Studio integration enforces JSON<→Record conversion instead of relying on rowStruct making number precision lost TCOMP-1542: Validator plugin uses family instead of pluginId (artifactId) to validate local-configuration TCOMP-1508: Don’t let Talend Starter Toolkit loose state on Enter in intellij TCOMP-1543: Add a uispec mapper TCOMP-1544: Update Geronimo JSON-P spec bundle to v1.3 TCOMP-1545: Update OpenWebBeans to version 2.0.12 TCOMP-1546: Update Meecrowave to 1.2.9 TCOMP-1547: Update Johnzon to 1.2.1 TCOMP-1279: Rewrite the pojo <→ record mapping to keep number types TCOMP-1504: Apache Beam 2.14.0 upgrade TCOMP-1505: Upgrade jackson-databind to 2.9.9.3 TCOMP-1506: Enable actions in bulk endpoint TCOMP-1507: Upgrade to johnzon 1.1.13 TCOMP-1511: Upgrade cxf to v3.3.3 TCOMP-1513: Upgrade to Tomcat 9.0.24 TCOMP-1514: Provide a RecordService to simplify record enrichment coding in processors TCOMP-1515: Record visitor API TCOMP-1517: Use netty 4.1.39.Final in junit HTTP tools TCOMP-1518: Upgrade to slf4j 1.7.28 TCOMP-1519: Upgrade to jib-core 0.10.1 TCOMP-1520: Don’t use JsonNode with Avro Fields anymore TCOMP-1521: Upgrade to Beam 2.15.0 TCOMP-1522: Basic singer/tap/stitch integration with kit components TCOMP-1523: Upgrade Apache Geronimo OpenAPI to v1.0.11 TCOMP-1524: Upgrade starter to gradle 5.6 TCOMP-1525: Upgrade commons-compress to v1.19 TCOMP-1527: Remove beam Mapper/Processor wrapping support TCOMP-1528: Upgrade to maven 3.6.2 TCOMP-1529: Asciidoctor 2.1.0 upgrade TCOMP-1530: geronimo-annotation 1.2 upgrade TCOMP-1532: Upgrade to Junit 5.5.2 TCOMP-1535: Upgrade to johnzon 1.2.0 TCOMP-1537: Upgrade to Tomcat 9.0.26 TCOMP-1538: Upgrade to jackson 2.9.10 TCOMP-1539: Rework default direct runner/spark classloader rules TCOMP-1540: Ensure Asciidoctor documentation rendering releases properly JRuby threads (main usage only) TCOMP-1478: /documentation/component/{id} internationalization does not work when embedded TCOMP-1479: When generating the documentation, it can happen the lang is wrong due to ResourceBundle usage TCOMP-1480: Servers docker images don’t have curl or wget available TCOMP-1497: POJO to Record mapping is not supported in processors TCOMP-1498: SVG2Mojo wrongly log the source file as being created TCOMP-1499: component-form does not support array of object of object if 2 levels use the same field name TCOMP-1500: Ensure component-form button have a key to have an id and propagate errors in the front TCOMP-1503: EnvironmentSecuredFilter not working on /environment/ TCOMP-1482: Enable web tester to switch the language TCOMP-1483: Enable to expose the documentation through the web tester TCOMP-1485: Asciidoctor documentation does not enable titles (component name and configuration ones) to be translated TCOMP-1486: Ensure locale mapping is configurable in component-server TCOMP-1484: Junit 5.5.0 upgrade TCOMP-1487: AsciidocMojo should only use ROOT locale by default TCOMP-1488: Enable to translate gridlayout names TCOMP-1489: Upgrade Tomcat to v9.0.22 TCOMP-1491: Upgrade JIB to v1.4.0 TCOMP-1492: Upgrade jackson-databind to 2.9.9.1 TCOMP-1493: Rewrite component exception to ensure they can be loaded after a serialization TCOMP-1494: Upgrade to junit jupiter 5.5.1 TCOMP-1495: Upgrade to Geronimo OpenAPI 1.0.10 TCOMP-1496: [testing tool] MainInputFactory does not support Record TCOMP-1501: Remove generate mojo TCOMP-1502: [maven plugin] upgrade jib-core to 0.10.0 TCOMP-1469: Studio maven repository not found OOTB TCOMP-1472: Connectors maven goal does not work in 1.1.10 TCOMP-1473: Docker image text log setup should use ISO8601 and not HH:mm:ss.SSS TCOMP-1470: Upgrade Tomcat to v9.0.21 TCOMP-1471: Upgrade Geronimo OpenAPI to v1.0.9 TCOMP-1474: Ensure proxies definition are java >=11 friendly TCOMP-1425: Spark classes not excluded anymore in component-runtime-beam leading to classloading issues TCOMP-1427: dependencies.txt mojo uses timestamped versions for snapshots instead of just -SNAPSHOT TCOMP-1431: [maven] Asciidoctor files should be attached with adoc extension and not jar one TCOMP-1433: [form-model] itemwidget ignored from uischema builder TCOMP-1438: Index cache can lead to invalid index list of component TCOMP-1440: Bulk components without @ElementListener when used with component-extension (default in the server) TCOMP-1441: Missing parameter init in the UiSchema Trigger builder TCOMP-1446: Rework gradle lifecycle TCOMP-1419: Upgrade build to groovy 2.5.7 TCOMP-1420: Upgrade maven compiler to 3.1.2 TCOMP-1422: Filter allowed beam classes in component-server image TCOMP-1423: Enable to customize studio maven repository for deploy-studio maven and gradle goal/task TCOMP-1426: Ensure Spark rule and @WithSpark uses a default version consistent with the runtime TCOMP-1430: Deprecate built-in icons in favor of vendor specific icons TCOMP-1432: basic dita generation for the component documentation TCOMP-1434: [form-model] Add withCondition to UISchema builder TCOMP-1435: Dont use beam_sdks_java_core shaded libraries TCOMP-1437: Add infinite metadata to ComponentDetail TCOMP-1444: Remove KnownJarsFilter since it is no more used to discover components TCOMP-1445: Icon must support SVG TCOMP-1448: [starter] provide a basic OpenAPI integration TCOMP-1449: Upgrade XBean to v4.14 TCOMP-1450: Add a read-only bulk endpoint in component-server TCOMP-1451: [upgrade] Johnzon 1.1.12 TCOMP-1452: [upgrade] Meecrowave 1.2.8 TCOMP-1453: Upgrade to CXF 3.3.2 TCOMP-1455: Prepare DateTime support in configurations TCOMP-1457: Upgrade to Apache Beam 2.13.0 TCOMP-1458: Ensure _placeholder presence is encouraged and validated TCOMP-1459: Experimental way to patch a component dependency TCOMP-1461: Extension API for the validator plugin TCOMP-1462: Validate through the corresponding build task provided SVG TCOMP-1464: Upgrade to OpenWebBeans 2.0.11 TCOMP-1465: Upgrade to JUnit 5.5.0-RC1 TCOMP-1466: Upgrade to ziplock 8.0.0-M2 TCOMP-1467: Upgrade mock server (testing tool) to netty 5.0.0.Alpha2 TCOMP-1468: Support docker-compose >= 1.23 in vault-proxy TCOMP-1374: ensure Utf8 avro strings don’t leak in AvroRecord API, even using get(Object.class, …) TCOMP-1375: When two sources use the same dataset and one source has additional required parameter the validation fails TCOMP-1384: Enhance studio guess schema algorithm to find implicitly the action to call if needed TCOMP-1388: Can’t change the dataset name in starter TCOMP-1389: Intellij starter fails to generate a project TCOMP-1398: Using after option of @updateable can lead to a null pointer exception in component-form TCOMP-1401: Documentation table is broken TCOMP-1407: Databricks: interface javax.json.stream.JsonGeneratorFactory is not visible from class loader TCOMP-1386: Add withRecord(String,Record) in Record.Builder TCOMP-1387: Use icon bundle version 3.1.0 TCOMP-1412: Add rest and couchbase icon to component api TCOMP-1376: Upgrade jupiter to 5.4.2 TCOMP-1385: talend.component.server.component.registry must be a list TCOMP-1390: Move component-api to component-runtime repository TCOMP-1392: Tomcat 9.0.19 upgrade TCOMP-1402: Provide a placeholder for classpath extensions in docker images TCOMP-1403: Upgrade asciidoctor to 2.0.0 and asciidoctor-pdf to alpha17 TCOMP-1404: Upgrade to Apache Beam 2.12.0 TCOMP-1408: Starter does not support types starting with a lowercase TCOMP-1411: ComponentManager relies on beam jar name. This is unlikely and should move to beam integration module. TCOMP-1417: Upgrade to Geronimo OpenAPI 1.0.8 TCOMP-1326: Avro Schema is not serializable as JSON so guess schema action does not work when compoennt-runtime-beam is present TCOMP-1330: Shade extensions don’t inherit from pluginrepositories TCOMP-1340: Tools webapp (talend-component:web) does not support changing the locale anymore TCOMP-1343: Use LogicalTypes.timestampMillis() on DATETIME for avro record builder TCOMP-1360: Renaming an option (@Option("custom")) does not work on fields of type object TCOMP-1370: ImageM2Mojo does not set timestamp in the docker image leading to component-server having a wrong lastUpdated value TCOMP-1372: Nested components don’t expose their doc deterministicly until it is overriden TCOMP-1341: Register deploy in studio task OOTB in gradle extension TCOMP-1325: Upgrade CXF to 3.3.1 TCOMP-1327: /environment iterates over deployed plugin for each call, this is not needed TCOMP-1328: Upgrade to Beam 2.11.0 TCOMP-1329: Lazy initialize parameter model to have a quicker cold start in plain main(String[]) TCOMP-1331: Use java 8u191 as base docker image TCOMP-1332: Provide a simple way to filter configurations and component on /index endpoints TCOMP-1334: Add a mojo to generate the list of components/services classes TCOMP-1335: Add in doc mojo table the type of configuration the parameter belongs to TCOMP-1336: Allow output processors to only have an @AfterGroup taking the list of record of the group in parameter TCOMP-1346: Upgrade to Tomcat 9.0.17 TCOMP-1347: Upgrade to Slf4j 1.7.26 TCOMP-1348: [form-core] Ensure suggestions trigger is bound to "change" event too TCOMP-1349: [form-core] When a tab is empty, don’t show it TCOMP-1350: talend.component.server.component.registry should support glob pattern TCOMP-1351: Upgrade jsoup for Spark Cluster Testing module TCOMP-1353: component-server must not use TALEND-INF/dependencies.txt but another path TCOMP-1354: Enforce services to belong to the delcaring service class TCOMP-1361: Upgrade to asciidoctorj 2.0.0-RC.1 TCOMP-1362: Beam Wrapped Components should throw shared exception types TCOMP-1366: Upgrade to XBean 4.13 to not track all classes scanned TCOMP-1371: Upgrade to Apache Geronimo OpenAPI 1.0.7 TCOMP-1307: support char and character types in configuration. TCOMP-1312: Component-form-core shouldn’t trigger validation of object due to conditional visibility (only individual fields are validable) TCOMP-1314: category field of the starter is broken TCOMP-1316: [build] Ensure snapshot use timestamped versions in dependencies.txt TCOMP-1306: Add RecordPointerFactory to enable to extract data from Record using json pointer spec TCOMP-1315: Ensure @Internationalized can use shortnames too in Messages.properties TCOMP-1303: Support docker configs/secrets in docker images TCOMP-1304: Vault proxy should support token configuration TCOMP-1305: Upgrade to beam 2.10.0 TCOMP-1308: Upgrade to Talend UI 2.6.0 TCOMP-1309: Upgrade to Component API 1.1.5 TCOMP-1310: Ensure there is a basic secured mecanism to store configuration data TCOMP-1317: Use Apache Geronimo Microprofile Config extensions (docker and secured string) TCOMP-1318: Upgrade to Apache Meecrowave 1.2.7 TCOMP-1319: Upgrade Apache Geronimo Metrics to 1.0.3 TCOMP-1320: Upgrade to Apache Geronimo OpenAPI 1.0.6 TCOMP-1321: Upgrade to Apache Geronimo OpenTracing 1.0.2 TCOMP-1322: Upgrade to Apache Geronimo Config 1.2.2 TCOMP-1263: When using @Updateable(after=xxx) the visibility condition (@ActiveIf) of the after field shouldn’t be inherited TCOMP-1264: AvroSchema does not unwrap null(able types) to map to Schema model TCOMP-1265: dataset / datastore cloud validation : allow nested configuration types TCOMP-1267: /documentation does not filter properly component TCOMP-1281: Add jackson-mapper-asl in docker image of the server TCOMP-1298: Support restricted lists for @Proposable TCOMP-1297: make max batch size property configurable for family and components through LocalConfiguration TCOMP-1266: Enhance starter to support dataset and datastore TCOMP-1268: Ensure /environment is not callable if not local or secured TCOMP-1269: Ensure ErrorReportValve does not leak Tomcat version OOTB TCOMP-1271: Upgrade to talend UI 2.3.0 TCOMP-1272: Move multiSelectTag to multiSelect for web environment TCOMP-1273: [build/dev plugin] Automatically open the browser for talend-component:web task/goal TCOMP-1276: Exclude xerces from component loadable resources for XMLReaderFactory TCOMP-1282: Upgrade meecrowave to 1.2.6 TCOMP-1283: Upgrade cxf to 3.3.0 TCOMP-1284: Upgrade to johnzon 1.1.11 TCOMP-1292: Provide a vault friendly integration for the server TCOMP-1293: Upgrade to Tomcat 9.0.16 TCOMP-1295: Ensure local-configuration.properties of a container are merged TCOMP-1296: Ensure user can enrich families with custom jar+configuration TCOMP-1245: Provided services (SPI) by tacokit not available TCOMP-1246: Rework docker image setup to use jib TCOMP-1247: Upgrade geronimo metrics to 1.0.2 TCOMP-1248: Upgrade to geronimo opentracing 1.0.3 TCOMP-1249: Provide segment extractor for doc endpoint TCOMP-1250: Make component documentation (@Documentation on component) i18n friendly TCOMP-1251: cache avrocoders used in SchemaRegistryCoder TCOMP-1252: Remove html support in documentation endpoint TCOMP-1253: Refine OpenAPI documentation TCOMP-1256: Add mapDescriptorToClassLoader to create a classloader from a list of gav TCOMP-1258: Support to build a Record from a provided Schema TCOMP-1259: Add getOptional to Record TCOMP-1223: byte[] not supported in AvroRecord (beam) TCOMP-1222: Ensure @WithComponents and @Environment are compatible TCOMP-1234: Upgrade to beam 2.9.0 TCOMP-1235: Upgrade to antora 2 TCOMP-1237: Upgrade component-api to 1.1.2 TCOMP-1238: Upgrade metrics and opentracing microprofile libraries in docker image to use Geronimo extensions TCOMP-1239: OpenWebBeans 2.0.9 upgrade TCOMP-1240: Johnzon 1.1.11 upgrade TCOMP-1242: Runtime validation error message wrongly interpolated TCOMP-1243: Ensure component classloader isolates the system classloader resources except for the JVM ones TCOMP-1170: [regression] HTTP testing module pom imports netty and jsonb stack TCOMP-1181: tacokit can’t pass the long type field from ui rightly TCOMP-1187: Job DSL does not support correctly parameters when they are URI/URL TCOMP-1189: Ensure primitive are not nullable in Record model (builder) TCOMP-1191: [beam] BeamIOTransformer does not support serialization of complex objects correctly TCOMP-1192: Ensure Avro schema union is interpreted as nullable in Record Schema model TCOMP-1194: [testing] Ensure BeamEnvironment adds component-runtime-beam TCOMP-1196: Nested maven repository not used for component module TCOMP-1197: Tacokit beam tests. NPE when creating the schema with RECORD type. TCOMP-1198: Tacokit beam tests. SchemaParseException ⇒ drop unsupported characters TCOMP-1200: Packages not defined from nested repository classes TCOMP-1201: includeTransitiveDependencies option of nested-maven-repository does not work TCOMP-1202: Refine avro classloading exclusion to accept hadoop and mapred packages TCOMP-1205: Empty JSon object lead to NPE TCOMP-1209: Ensure SerializableCoder is replaced with a contextual version to support Talend Component Kit classloading model TCOMP-1210: BeamComponentExtension should let the exception go back to the caller when the transform fails TCOMP-1215: Nested maven repository in jars don’t go through transformers TCOMP-1218: Record entries order shouldn’t be sorted by the runtime TCOMP-1185: Support maxBatchSize in Job test runner for standalone mode TCOMP-1171: Remove component proxy server from the project TCOMP-1182: Ensure the property editor for the configuration registers the default converters TCOMP-1183: Upgrade JRuby to 9.2.4.0 TCOMP-1184: Avoid to do a group by key in BeamExecutor (job DSL) when not needed TCOMP-1188: Tolerate null for dates in Records TCOMP-1190: Enable secure processing for DocumentBuilderFactory instances TCOMP-1193: Add injectable ContainerInfo with the containerId (plugin) in services TCOMP-1195: Enable user to extend BeamEnvironment test tempalte more easily TCOMP-1199: Nested repository not used when the classpath is not composed of a single jar TCOMP-1204: [dependency upgrade] XBean 4.12 TCOMP-1207: [beam] add ContextualSerializableCoder TCOMP-1213: Upgrade guava to v27 for testing modules TCOMP-1216: Take into account the visibility for the parameter validation TCOMP-1217: Add JVM system property talend.component.runtime.serialization.java.inputstream.whitelist for our custom object input stream TCOMP-1219: Upgrade starter to gradle 5 TCOMP-1220: Upgrade Maven to 3.6.0 in starter TCOMP-1121: [tacokit proxy] suggestion trigger creation issue TCOMP-1122: [tacokit proxy] slefRefrence filter configuration type by name, type and family TCOMP-1123: Processor component onNext duplicate columns in record for rowStructs TCOMP-1126: UiSpecService shouldn’t show the documentation by default TCOMP-1129: form core - $selfReference breaks triggers TCOMP-1130: component form - default value of maxBatchSize prop loose it type. TCOMP-1131: [beam integration] Ensure Coder is contextual (classloader) TCOMP-1132: Ensure beam custom Coders implement equals.hashCode for beam contract TCOMP-1148: Asciidoctor documentation fails for collection of objects TCOMP-1149: [testing] BeamEnvironment does not reset PipelineOptionsFactory properly for beam > 2.4 TCOMP-1155: [proxy server] arrays not supporting null values in ConfigurationFormatter TCOMP-1159: AvroSchema does not support DATETTIME type (beam module) TCOMP-1168: Avro record implementation ignores nullable/union TCOMP-1143: Ensure icons are validated and fail the build if a custom one is missing (validate mojo) TCOMP-1112: Let beam PTransform define an @ElementListener method to set the component design (inputs/outputs) TCOMP-1113: Simplify the scanning by assuming there is a TALEND-INF/dependencies.txt in components TCOMP-1120: BeamMapperImpl.isStream not accurate for UnboundedSource TCOMP-1124: Add /metrics endpoint TCOMP-1125: Extend CustomPropertyConverter to pass the convertion context TCOMP-1127: Record doesn’t support null values TCOMP-1133: CXF 3.2.7 upgrade TCOMP-1134: Ensure any input/output have a dataset TCOMP-1135: Ensure any dataset has a datastore TCOMP-1136: deprecate "generate" mojo TCOMP-1145: [dependency upgrade] Beam 2.8.0 TCOMP-1146: implement infinite=true in PartitionMapper/Input TCOMP-1150: Upgrade rat plugin to 0.13 TCOMP-1154: Required validation at runtime ignores lists and nested objects TCOMP-1157: [dependency upgrade] Tomcat 9.0.13 TCOMP-1158: Enable JUnit test collector to use a static storage instead of thread related one TCOMP-1160: Upgrade spark to 2.4.0 TCOMP-1161: Upgrade shade plugin to 3.2.1 TCOMP-1162: Upgrade nested-maven-repository shade transformers to support last maven versions TCOMP-1163: Upgrade openwebbeans to 2.0.8 TCOMP-1164: Validate mojo does not log any success information TCOMP-1165: Dependency mojo does not log any success information TCOMP-1166: Documentation mojo does not log generated files properly TCOMP-1167: Beam-Avro record name generation should use avro fingerprint to be more unique than current logic TCOMP-1086: Fix documentation about DiscoverSchema TCOMP-1064: Update action can’t receive List parameter TCOMP-1110: When a configuration has no layout and uses @AfterGroup the configuration is lost TCOMP-1111: Move to PropertyEditorRegistry from xbean instead of using the deprecated static class TCOMP-1000: @Option name value is not respected on fields TCOMP-1008: Enum order is lost TCOMP-1009: (web) OptionsOrder ignored for tables (List), fields located in random order TCOMP-1028: [tools-webapp] submit button no more functional TCOMP-1031: DiscoverSchema parameters are not correctly mapped in Studio GuessSchema runtime TCOMP-1044: Fix java.lang.ClassCastException in TableActionParameter TCOMP-1046: String option can’t set default value from a file TCOMP-1056: ActiveIf doesn’t work in advanced settings TCOMP-1072: Metadata migration issues TCOMP-1074: talend-component mvn plugin : deploy-in-studio need to rise an error when component is already installed TCOMP-1075: component reload file on windows after deploying a modified jar TCOMP-1076: component starter - fix mapper generation (Record integration) TCOMP-1077: component starter - ensure kit version are updated atomically. TCOMP-1078: Guess Schema button is not shown on Basic Settings view TCOMP-1082: Fix Exception during HealthCheck parameter deserialization TCOMP-1085: [classloader] com.sun is too wide as exclusion TCOMP-1104: Fix drag and drop issue for dataset/datastore metadata TCOMP-779: Drop down list Java type in configuration class TCOMP-819: Processor doesn’t produce more than 1 row on each iteration TCOMP-917: Migration handler need only to receive component configuration TCOMP-941: Default and init values are ignored in connection wizzard (datastore/dataset) TCOMP-968: Trigger AsyncValidation call only when option annotated with Validable is changed TCOMP-970: Add support for complex parameter types for AsyncValidation methods TCOMP-973: component migration - the configuration version need to be serialized in addition to the version of the component TCOMP-984: Integrate ParameterizedTest with component-runtime-HTTP-junit capture mode TCOMP-988: component migration - fix nested configuration migration TCOMP-989: .car studio install command breaks config.ini of the studio TCOMP-991: metadat : ignore activations from config not being part of the form while creating metadata TCOMP-996: metadata : migration issues TCOMP-1001: [proxy] ConfigurationClient should expose a migrate method TCOMP-1011: Ensure datastore/dataset i18n names are validated by the maven/gradle plugins TCOMP-1013: Add an operator support in @ActiveIfs (OR/AND switch) TCOMP-1014: Ensure a dataset has a source which has no other required parameters in the validator TCOMP-1029: Extend ActiveIf EvaluationStrategy with CONTAINS strategy TCOMP-1063: Integrate Record API to the studio TCOMP-1069: restrict input branches for output components to only one. TCOMP-1071: support actions i18n display name TCOMP-1092: Ensure @Configuration POJO are injectable as Supplier in services TCOMP-1094: Add FullSerializationRecordCoder coder for Record in beam module TCOMP-1095: Ensure all configuration type models root entries are named "configuration" TCOMP-993: [proxy] Propagate UiSpecContext in referenceservice#findByTypeAndName TCOMP-994: [dependency upgrade] CXF 3.2.6 TCOMP-1003: [dependency upgrade] Tomcat 9.0.12 TCOMP-1004: [dependency upgrade] Log4j2 2.11.1 TCOMP-1015: Upgrade icons to 1.0.0 TCOMP-1019: (form) enum should lead to restricted datalist TCOMP-1037: [dependency upgrade] Johnzon 1.1.9 TCOMP-1038: Drop spring client from component-form-core TCOMP-1041: HTTPClient should enable to process InputStream directly TCOMP-1042: Upgrade to JUnit 5.3.1 TCOMP-1045: Add documentation in metadata and enable to use it in the UI on configuration TCOMP-1047: Make Suggestable text field editable (align with web) TCOMP-1048: Add update API for configuration TCOMP-1049: Add completion support for actions displayname in intellij plugin TCOMP-1050: Provide simple OAuth1 integration TCOMP-1051: Remove brave and move to geronimo-opentracing TCOMP-1054: Introduce @Configuration API TCOMP-1055: remove the ExecutionResource TCOMP-1057: Add ActiveIf on @Proposable test-case TCOMP-1058: Add DefaultValue on proposable/dynamicValue testcase TCOMP-1059: Rework generic record format TCOMP-1073: [maven/gradle plugin] Add configuration support in web goal TCOMP-1079: Document new Record structure TCOMP-1080: [dependency upgrade] Meecrowave 1.2.4 TCOMP-1081: ComponentManager should ignore engine classes in its filtering TCOMP-1087: Jsonb service should serialize byte[] as BASE64 TCOMP-1089: [starter] Upgrade gradle to 4.10.2 TCOMP-1090: [form] Main/Advanced order not respected when some remote action are involved TCOMP-1091: Ensure main component is preferred over test ones in a maven project TCOMP-1093: [dependency upgrade] netty 4.1.30.Final for junit HTTP testing module TCOMP-1096: [dependency upgrade] xbean 4.10 TCOMP-1097: [dependency upgrade] Beam 2.7.0 TCOMP-1099: Upgrade web ui bundle to 1.0.2 TCOMP-1101: Add conditional rendering in the generated documentation TCOMP-1102: Reflect in documentation that Validable/AsyncValidation doesn’t support Object types TCOMP-1106: Enable to generate the component documentation in multiple languages TCOMP-1107: ConfigurableClassLoader does not priviledges container classloader for getResourceAsStream TCOMP-877: [documentation] Sample implementation of bulk/batch/commit-interval using groups TCOMP-980: Provide a ValidationService in server-proxy TCOMP-985: Align docker git metada on out Standard TCOMP-998: [dependency upgrade] Apache Commons Compress 1.18 TCOMP-911: Suggestions callback doesn’t support Configuration parameters TCOMP-921: String cannot be cast to Boolean when adding table with checkboxes TCOMP-922: component manager : support loading dependencies from job lib folder. TCOMP-924: component-kit.js errors are not sent to the error handler TCOMP-927: talend-component:web errors are not always unwrapped and understandable TCOMP-934: Ensure Studio rely on category and doesn’t append family name TCOMP-960: Suggestions parameters are not correctly resolved in Studio TCOMP-961: Default value of Suggestions method parameter is ignored TCOMP-964: ClassCastException is thrown when non-string values are used as Suggestions method parameter TCOMP-825: Provide component server proxy TCOMP-928: Add negate and evaluation strategy to @ActiveIf TCOMP-929: Ensure category contains the family TCOMP-816: Check migration feature and implement missing use-cases TCOMP-918: create a mvn bom with tacokit stack to keep some dependencies aligned between component-runtime and it’s studio integration TCOMP-932: Avoid Kafka recursive logging for component server TCOMP-933: Drop component-kit.js module TCOMP-935: Component server should log application and service in kafka mode TCOMP-938: Add a builtin::HTTP trigger in the server proxy TCOMP-939: Ensure the proxy server can lookup references with a SPI TCOMP-943: (web) Grand parent references for triggers not well resolved TCOMP-944: (proxy server) Ensure the trigger are well resolved for references TCOMP-947: (maven/gradle) ensure web task logs there is a UI TCOMP-953: Upgrade to ziplock 7.0.5 TCOMP-954: Upgrade netty to 4.1.28.Final for the test stack TCOMP-958: Componentvalidator error message in case of an unsupported type is misleading TCOMP-959: [dependency upgrade] Upgrade to icon bundle 0.202.0 TCOMP-962: .car deploy-in-studio command (CarMain) should support to override an existing version TCOMP-965: [dependency upgrade] Apache Beam 2.6.0 TCOMP-966: Ensure Studio integration renames HTTP threads to identify them more explicitly TCOMP-967: Ensure parameter index is in metadata for services and constructors TCOMP-919: Starter doesn’t synchronize correctly with central versions TCOMP-920: Use Meecrowave 1.2.3 TCOMP-888: Designer pipeline records counter are wrong for tacokit components with multiples outputs TCOMP-899: Update Beam 2.5.0 compatibility TCOMP-903: [tacokit studio integration] - Guess schema - better handling of number types recognition TCOMP-904: [tacokit studio integration] - fix job classpath generation TCOMP-913: Fix absolute path resolution for child of child use-case TCOMP-900: [tacokit studio integration] - Handle conditional outputs TCOMP-898: Ensure starter will be able to auto update its versions to avoid redeployments TCOMP-905: Enrich scanning exclusion set TCOMP-906: Minimalist JsonObject to IndexeredRecord utilities for beam TCOMP-907: Support maxBatchSize as in the studio in Beam TCOMP-910: Add maxbatchsize as built in parameter to Processor meta model TCOMP-915: Upgrade Apache Meecrowave to 1.2.2 TCOMP-822: [Windows] deploy-in-studio & car copy jar command in mvn plugin - don’t work if the studio is running TCOMP-844: Service default method forwarded to interface method instead of implementation one if exists TCOMP-848: [junit5] implicit mock collector and emitter are not resetted per method TCOMP-851: [form] UiSchema shouldn’t have a JsonSchema TCOMP-858: @OptionsOrder not respected by form-core TCOMP-862: [form-core] ".." path is not correctly resolved TCOMP-863: Job DSL doesn’t support multiple outputs TCOMP-873: Fix shade junit-HTTP module : remove shaded dependencies from generated artifact TCOMP-889: [form] arrays are lost in trigger paths TCOMP-890: Merge the component outputs (by name) from @AfterGroup and @ElementListener TCOMP-893: Don’t log a warning for services when parameters don’t have i18n support TCOMP-834: Ensure that component has only one configuration argument. TCOMP-845: [junit] ComponentsHandler misses findService TCOMP-846: [junit] allow to inject current plugin services in test class TCOMP-847: Support gzip in JUnit HTTP tooling TCOMP-849: [junit HTTP] support to match the request payload TCOMP-850: MavenDecrypter should tolerate ${env.xxx} syntax TCOMP-861: Ensure Car Mojo can be skipped TCOMP-887: [studio] add chunk size advanced common param for processors & output TCOMP-892: Validate runtime configuration before executing the runtime TCOMP-829: Configuration Type tree is not correctly computed TCOMP-830: Move all configuration to Microprofile Config instead of DeltaSpike TCOMP-832: Provide a way to access lastUpdatedTimestamp in rest api TCOMP-833: Upgrade gradle+maven for the starter TCOMP-839: Add an API to load lazily the potential values of a list TCOMP-840: Upgrade icon bundle to 0.190.2 TCOMP-841: Add validation of option names in the validator TCOMP-852: [dependency upgrade] Upgrade shrinkwrap-resolver-impl-maven to 3.1.3 TCOMP-855: Support service injections in services TCOMP-856: [dependency upgrade] OpenWebBeans 2.0.6 TCOMP-857: SimpleCollector must not depend on junit 4 TCOMP-864: Mojo should be thread safe for car/dependencies.txt generation TCOMP-867: Expose Injector service TCOMP-868: Create an ObjectFactory service TCOMP-869: Ensure actions can get injected the requested lang TCOMP-870: Provide Beam DoFn to simplify the migration from IndexedRecord to JsonObject TCOMP-876: Allow custom converters in form-core TCOMP-878: Add beam in the docker image OOTB TCOMP-879: CarMojo doesn’t use car extension to attach the artifact TCOMP-880: [dependency upgrade] Maven 3.5.4 TCOMP-881: [dependency upgrade] CXF 3.2.5 TCOMP-882: [dependency upgrade] Tomcat 9.0.10 TCOMP-883: [dependency upgrade] Beam 2.5.0 TCOMP-884: [dependency upgrade] Upgrade to icon bundle 0.197.0 TCOMP-894: [dependency upgrade] Johnzon 1.1.8 TCOMP-895: [dependency upgrade] xbean 4.9 TCOMP-827: Fix Automatic-Module-Name TCOMP-811: Upgrade to tomcat 9.0.8 TCOMP-826: Extract component model from component server to a new artifact TCOMP-763: Add a dev mode in the studio for tacokit TCOMP-802: Add method to upload dependencies from .car to nexus TCOMP-808: Upgrade to JUnit 5.2.0 TCOMP-809: compress js and css for the starter TCOMP-810: ui spec service uses a multiselecttag for a proposable on a string field TCOMP-804: Idea plugin doesn’t render properly configuration inputs TCOMP-798: intellij plugin - add official starter url TCOMP-799: @Checkable expects the datastore name to match the validation name TCOMP-806: Ensure server and starter support gzip TCOMP-643: Deployment TCOMP-770: Removing component from web UI causes wrong number of components in summary TCOMP-775: Starter - Fix properties keys generation TCOMP-776: component-kit.js ignore credentials TCOMP-783: ActiveIfs doesn’t make option visible TCOMP-796: Datastore check (@Checkable) should default meta parameters to "datastore" if none is found TCOMP-773: Extend the HTTP client api to handle more generic use cases TCOMP-771: ConfigurableClassLoader should skip scala.* classes TCOMP-772: Upgrade icon set to ui/icons 0.179.0 TCOMP-774: Upgrade xbean to 4.8 TCOMP-768: More tolerance of configuration prefix for implicit migration of configuration node in form core library TCOMP-756: Setup maven clirr plugin for component-api +testing TCOMP-762: Starter should only propose a single category level in the ui TCOMP-767: Ensure the configurationtype endpoints have matching name/path values TCOMP-761: Merge component-runtime-manager and component-runtime-standalone TCOMP-764: Clean up component-form-core dependencies TCOMP-765: Upgrade to batik 1.9.1 TCOMP-752: Fix Advanced settings and Test connection button appearance in repository wizard TCOMP-757: Duplicate method name "writeReplace" with signature "()Ljava.lang.Object;" in class file TCOMP-751: Support gzip compression on component-server TCOMP-753: Make classpath scanning to find component configurable TCOMP-758: Support component-server server configuration from system properties TCOMP-759: Enum must be i18n TCOMP-738: Component Server should respect ~/.m2/settings.xml local repository if it exists TCOMP-739: SerializationTransformer shouldn’t use ComponentManager to avoid ClassNotFoundException TCOMP-740: UISpecService should be reactive and use a CompletionStage based API TCOMP-741: UISpecService configuration support TCOMP-742: Configuration Type properties should be rooted TCOMP-744: Ensure wrapped BeamIO uses the right TCCL TCOMP-745: [Dependency Upgrade] CXF 3.2.4 TCOMP-746: [Dependency Upgrade] Tomcat 9.0.6 TCOMP-747: [Dependency Upgrade] Log4j2 2.11.0 TCOMP-748: Make configurationtype index endpoint lighter OOTB TCOMP-749: Intellij Idea plugin TCOMP-750: Unify @Pattern using javascript regex instead of a mixed mode TCOMP-734: Add support for context and globalMap values in Tacokit component settings TCOMP-733: support to use a beam pipeline under the hood for beam components in di TCOMP-693: Integrate Migration API TCOMP-737: upgrade to beam 2.4.0 TCOMP-731: Configuration Type migration handler skipped TCOMP-725: MavenDecrypter doesn’t support comments in settings.xml TCOMP-726: When a component is not found the error message can be misleading TCOMP-728: HTTP client doesn’t ignore empty query parameters TCOMP-722: WebSocket connection fails with a NPE when the endpoint doesn’t exists TCOMP-723: Adding configurationByExample utility to create query string for Job DSL TCOMP-724: Documentation endpoint doesn’t support HTML TCOMP-446: Support Embedded Documentation TCOMP-650: Ensure component can be executed in beam pipelines TCOMP-651: Ensure beam components can be wrapped and used through the Talend Component Kit Framework TCOMP-653: Web Form metamodel service TCOMP-655: Catalog service TCOMP-656: UISpec compatibility TCOMP-658: Add test Source/Sink collectors in JUnit integration TCOMP-659: Basic job builder API to simplify JUnit tests TCOMP-662: Validation Mojo TCOMP-664: Local testing server for dev TCOMP-675: Select a communication solution for Talend Component Kit server TCOMP-680: Register components into the Studio Palette TCOMP-681: Studio parameters form integration TCOMP-682: Studio Metadata integration TCOMP-683: Studio Runtime integration TCOMP-691: Create context menu for Tacokit node in repository panel TCOMP-719: Support Input Definition TCOMP-720: Support Output Definition TCOMP-721: Initial Widget Definitions

This tutorial shows how to create components that consume a REST API. The component developed as example in this tutorial is an input component that provides a search functionality for Zendesk using its Search API. Lombok is used to avoid writing getter, setter and constructor methods. You can generate a project using the Talend Components Kit starter, as described in this tutorial. The input component relies on Zendesk Search API and requires an HTTP client to consume it. The Zendesk Search API takes the following parameters on the /api/v2/search.json endpoint. query : The search query. sort_by : The sorting type of the query result. Possible values are updated_at, created_at, priority, status, ticket_type, or relevance. It defaults to relevance. sort_order: The sorting order of the query result. Possible values are asc (for ascending) or desc (for descending). It defaults to desc. Talend Component Kit provides a built-in service to create an easy-to-use HTTP client in a declarative manner, using Java annotations. No additional implementation is needed for the interface, as it is provided by the component framework, according to what is defined above. This HTTP client can be injected into a mapper or a processor to perform HTTP requests. This example uses the basic authentication that supported by the API. The first step is to set up the configuration for the basic authentication. To be able to consume the Search API, the Zendesk instance URL, the username and the password are needed. The data store is now configured. It provides a basic authentication token. Once the data store is configured, you can define the dataset by configuring the search query. It is that query that defines the records processed by the input component. Your component is configured. You can now create the component logic. Mappers defined with this tutorial don’t implement the split part because HTTP calls are not split on many workers in this case. Once the component logic implemented, you can create the source in charge of performing the HTTP request to the search API and converting the result to JsonObject records. You now have created a simple Talend component that consumes a REST API. To learn how to test this component, refer to this tutorial.

Apache addict I'm involved in several project (OpenWebBeans, Johnzon, Geronimo, Meecrowave, ... - HTTP://home.apache.org/committer-index.html#rmannibucau). Blog: HTTPs://rmannibucau.metawerx.net Software engineer @Talend. Components team member. Blog: undx.github.io Technical Writer Frontend Architect R&D Frontend Architect. This is my Talend account. You can check out @jsomsanith for my personal account Senior principal software engineer at @Talend, security contributor at @apache. Blog: HTTP://coheigea.blogspot.com/ Committer and PMC member of Apache Beam and Apache Avro. Free education and Open Source enthusiast. Distributed Systems practitioner (victim?) Blog: HTTPs://ismaelmejia.com/ Templar Blog: HTTPs://www.linkedin.com/in/zoltantakacsdev/ Focused on Big Data technologies. Open Source fan, I contribute to Apache projects. I'm an Apache Beam committer and PMC member Blog: HTTPs://echauchot.blogspot.com/ Senior Cloud Software Architect Product Manager, Information Architect Blog: www.talend.com 573PH4N3 D3M15 K3RM480N

Testing code that consumes REST APIs can sometimes present many constraints: API rate limit, authentication token and password sharing, API availability, sandbox expiration, API costs, and so on. As a developer, it becomes critical to avoid those constraints and to be able to easily mock the API response. The component framework provides an API simulation tool that makes it easy to write unit tests. This tutorial shows how to use this tool in unit tests. As a starting point, the tutorial uses the component that consumes Zendesk Search API and that was created in a previous tutorial. The goal is to add unit tests for it. For this tutorial, four tickets that have the open status have been added to the Zendesk test instance used in the tests. To learn more about the testing methodology used in this tutorial, refer to Component JUnit testing. Create a unit test that performs a real HTTP request to the Zendesk Search API instance. You can learn how to create a simple unit test in this tutorial. the authentication configuration using Zendesk instance URL and credentials. the search query configuration to get all the open ticket, ordered by creation date and sorted in descending order. The test is now complete and working. It performs a real HTTP request to the Zendesk instance. As an alternative, you can use mock results to avoid performing HTTP requests every time on the development environment. The real HTTP requests would, for example, only be performed on an integration environment. To transform the unit test into a mocked test that uses a mocked response of the Zendesk Search API: Add the two following JUnit rules provided by the component framework. JUnit4HTTPApi: This rule starts a simulation server that acts as a proxy and catches all the HTTP requests performed in the tests. This simulation server has two modes : capture : This mode forwards the captured HTTP request to the real server and captures the response. simulation : this mode returns a mocked response from the responses already captured. This rule needs to be added as a class rule. JUnit4HTTPApi: This rule has a reference to the first rule. Its role is to configure the simulation server for every unit test. It passes the context of the running test to the simulation server. This rule needs to be added as a simple (method) rule. Example to run in a simulation mode: Make the test run in capture mode to catch the real API responses that can be used later in the simulated mode. To do that, set a new talend.junit.HTTP.capture environment variable to true. This tells the simulation server to run in a capture mode. The captured response is saved in the resources/talend.testing.HTTP package in a JSON format, then reused to perform the API simulation.

Refer to the following links to access the detailed API reference documentation: API Documentation JUnit API Documentation HTTP JUnit API Documentation

Learn about the Component Server with the following articles: Component server and HTTP API Component Server Vault Proxy

The Component Kit Starter lets you design your components configuration and generates a ready-to-implement project structure. The Starter is available on the web or as an IntelliJ plugin. This tutorial shows you how to use the Component Kit Starter to generate new components for MySQL databases. Before starting, make sure that you have correctly setup your environment. See this section. When defining a project using the Starter, do not refresh the page to avoid losing your configuration. Before being able to create components, you need to define the general settings of the project: Create a folder on your local machine to store the resource files of the component you want to create. For example, C:/my_components. Open the Starter in the web browser of your choice. Select your build tool. This tutorial uses Maven, but you can select Gradle instead. Add any facet you need. For example, add the Talend Component Kit Testing facet to your project to automatically generate unit tests for the components created in the project. Enter the Component Family of the components you want to develop in the project. This name must be a valid java name and is recommended to be capitalized, for example 'MySQL'. Once you have implemented your components in the Studio, this name is displayed in the Palette to group all of the MySQL-related components you develop, and is also part of your component name. Select the Category of the components you want to create in the current project. As MySQL is a kind of database, select Databases in this tutorial. This Databases category is used and displayed as the parent family of the MySQL group in the Palette of the Studio. Complete the project metadata by entering the Group, Artifact and Package. By default, you can only create processors. If you need to create Input or Output components, select Activate IO. By doing this: Two new menu entries let you add datasets and datastores to your project, as they are required for input and output components. Input and Output components without dataset (itself containing a datastore) will not pass the validation step when building the components. Learn more about datasets and datastores in this document. An Input component and an Output component are automatically added to your project and ready to be configured. Components added to the project using Add A Component can now be processors, input or output components. A datastore represents the data needed by an input or output component to connect to a database. When building a component, the validateDataSet validation checks that each input or output (processor without output branch) component uses a dataset and that this dataset has a datastore. You can define one or several datastores if you have selected the Activate IO step. Select Datastore. The list of datastores opens. By default, a datastore is already open but not configured. You can configure it or create a new one using Add new Datastore. Specify the name of the datastore. Modify the default value to a meaningful name for your project. This name must be a valid Java name as it will represent the datastore class in your project. It is a good practice to start it with an uppercase letter. Edit the datastore configuration. Parameter names must be valid Java names. Use lower case as much as possible. A typical configuration includes connection details to a database: url username password. Save the datastore configuration. A dataset represents the data coming from or sent to a database and needed by input and output components to operate. The validateDataSet validation checks that each input or output (processor without output branch) component uses a dataset and that this dataset has a datastore. You can define one or several datasets if you have selected the Activate IO step. Select Dataset. The list of datasets opens. By default, a dataset is already open but not configured. You can configure it or create a new one using the Add new Dataset button. Specify the name of the dataset. Modify the default value to a meaningful name for your project. This name must be a valid Java name as it will represent the dataset class in your project. It is a good practice to start it with an uppercase letter. Edit the dataset configuration. Parameter names must be valid Java names. Use lower case as much as possible. A typical configuration includes details of the data to retrieve: Datastore to use (that contains the connection details to the database) table name data Save the dataset configuration. To create an input component, make sure you have selected Activate IO. When clicking Add A Component in the Starter, a new step allows you to define a new component in your project. The intent in this tutorial is to create an input component that connects to a MySQL database, executes a SQL query and gets the result. Choose the component type. Input in this case. Enter the component name. For example, MySQLInput. Click Configuration model. This button lets you specify the required configuration for the component. By default, a dataset is already specified. For each parameter that you need to add, click the (+) button on the right panel. Enter the parameter name and choose its type then click the tick button to save the changes. In this tutorial, to be able to execute a SQL query on the Input MySQL database, the configuration requires the following parameters:+ a dataset (which contains the datastore with the connection information) a timeout parameter. Closing the configuration panel on the right does not delete your configuration. However, refreshing the page resets the configuration. Specify whether the component issues a stream or not. In this tutorial, the MySQL input component created is an ordinary (non streaming) component. In this case, leave the Stream option disabled. Select the Record Type generated by the component. In this tutorial, select Generic because the component is designed to generate records in the default Record format. You can also select Custom to define a POJO that represents your records. Your input component is now defined. You can add another component or generate and download your project. When clicking Add A Component in the Starter, a new step allows you to define a new component in your project. The intent in this tutorial is to create a simple processor component that receives a record, logs it and returns it at it is. If you did not select Activate IO, all new components you add to the project are processors by default. If you selected Activate IO, you can choose the component type. In this case, to create a Processor component, you have to manually add at least one output. If required, choose the component type: Processor in this case. Enter the component name. For example, RecordLogger, as the processor created in this tutorial logs the records. Specify the Configuration Model of the component. In this tutorial, the component doesn’t need any specific configuration. Skip this step. Define the Input(s) of the component. For each input that you need to define, click Add Input. In this tutorial, only one input is needed to receive the record to log. Click the input name to access its configuration. You can change the name of the input and define its structure using a POJO. If you added several inputs, repeat this step for each one of them. The input in this tutorial is a generic record. Enable the Generic option and click Save. Define the Output(s) of the component. For each output that you need to define, click Add Output. The first output must be named MAIN. In this tutorial, only one generic output is needed to return the received record. Outputs can be configured the same way as inputs (see previous steps). You can define a reject output connection by naming it REJECT. This naming is used by Talend applications to automatically set the connection type to Reject. Your processor component is now defined. You can add another component or generate and download your project. To create an output component, make sure you have selected Activate IO. When clicking Add A Component in the Starter, a new step allows you to define a new component in your project. The intent in this tutorial is to create an output component that receives a record and inserts it into a MySQL database table. Output components are Processors without any output. In other words, the output is a processor that does not produce any records. Choose the component type. Output in this case. Enter the component name. For example, MySQLOutput. Click Configuration Model. This button lets you specify the required configuration for the component. By default, a dataset is already specified. For each parameter that you need to add, click the (+) button on the right panel. Enter the name and choose the type of the parameter, then click the tick button to save the changes. In this tutorial, to be able to insert a record in the output MySQL database, the configuration requires the following parameters:+ a dataset (which contains the datastore with the connection information) a timeout parameter. Closing the configuration panel on the right does not delete your configuration. However, refreshing the page resets the configuration. Define the Input(s) of the component. For each input that you need to define, click Add Input. In this tutorial, only one input is needed. Click the input name to access its configuration. You can change the name of the input and define its structure using a POJO. If you added several inputs, repeat this step for each one of them. The input in this tutorial is a generic record. Enable the Generic option and click Save. Do not create any output because the component does not produce any record. This is the only difference between an output an a processor component. Your output component is now defined. You can add another component or generate and download your project. Once your project is configured and all the components you need are created, you can generate and download the final project. In this tutorial, the project was configured and three components of different types (input, processor and output) have been defined. Click Finish on the left panel. You are redirected to a page that summarizes the project. On the left panel, you can also see all the components that you added to the project. Generate the project using one of the two options available: Download it locally as a ZIP file using the Download as ZIP button. Create a GitHub repository and push the project to it using the Create on Github button. In this tutorial, the project is downloaded to the local machine as a ZIP file. Once the package is available on your machine, you can compile it using the build tool selected when configuring the project. In the tutorial, Maven is the build tool selected for the project. In the project directory, execute the mvn package command. If you don’t have Maven installed on your machine, you can use the Maven wrapper provided in the generated project, by executing the ./mvnw package command. If you have created a Gradle project, you can compile it using the gradle build command or using the Gradle wrapper: ./gradlew build. The generated project code contains documentation that can guide and help you implementing the component logic. Import the project to your favorite IDE to start the implementation. The Component Kit Starter allows you to generate a component development project from an OpenAPI JSON descriptor. Open the Starter in the web browser of your choice. Enable the OpenAPI mode using the toggle in the header. Go to the API menu. Paste the OpenAPI JSON descriptor in the right part of the screen. All the described endpoints are detected. Unselect the endpoints that you do not want to use in the future components. By default, all detected endpoints are selected. Go to the Finish menu. Download the project. When exploring the project generated from an OpenAPI descriptor, you can notice the following elements: sources the API dataset an HTTP client for the API a connection folder containing the component configuration. By default, the configuration is only made of a simple datastore with a baseUrl parameter.

This tutorial shows how to adapt the test configuration of the Zendesk search component that was done in this previous tutorial to make it work in a Continuous Integration environment. In the test, the Zendesk credentials are used directly in the code to perform a first capture of the API response. Then, fake credentials are used in the simulation mode because the real API is not called anymore. However, in some cases, you can require to continue calling the real API on a CI server or on a specific environment. To do that, you can adapt the test to get the credentials depending on the execution mode (simulation/passthrough). These instructions concern the CI server or on any environment that requires real credentials. This tutorial uses: A Maven server that supports password encryption as a credential provider. Encryption is optional but recommended. The MavenDecrypterRule test rule provided by the framework. This rule lets you get credentials from Maven settings using a server ID. To create encrypted server credentials for the Zendesk instance: Create a master password using the command: mvn --encrypt-master-password . Store this master password in the settings-security.xml file of the ~/.m2 folder. Encrypt the Zendesk instance password using the command: mvn --encrypt-password . Create a server entry under servers in Maven settings.xml file located in the ~/.m2 folder. You can store the settings-security.xml and settings.xml files elsewhere that the default location (~/.m2). To do that, set the path of the directory containing the files in the talend.maven.decrypter.m2.location environment variable. Add the MavenDecrypterRule rule to the test class. This rule allows to inject server information stored in Maven settings.xml file to the test. The rule also decrypts credentials if they are encrypted. Inject the Zendesk server to the test. To do that, add a new field to the class with the @DecryptedServer annotation, that holds the server ID to be injected. The MavenDecrypterRule is able to inject the server instance into this class at runtime. The server instance contains the username and the decrypted password. Use the server instance in the test to get the real credentials in a secured manner. Once modified, the complete test class looks as follows: This test will continue to work in simulation mode, because the API simulation proxy is activated. This tutorial shows how to set up a CI server in passthrough mode using Jenkins. Log in to Jenkins. Click New Item to create a new build job. Enter an Item name (Job name) and choose the freestyle job. Then click OK. In the Source Code Management section, enter your project repository URL. A GitHub repository is used in this tutorial. Specify the master branch as Branches to build. In the Build section, click Add build step and choose Invoke top-level Maven targets. Choose your Maven version and enter the Maven build command. In this case: clean install. Then, click Save. The -Dtalend.junit.HTTP.passthrough=true option is part of the build command. This option tells the API simulation proxy to run in passthrough mode. This way, all the HTTP requests made in the test are forwarded to the real API server. The MavenDecrypterRule rule allows to get the real credentials. You can configure the passthrough mode globally on your CI server by setting the talend.junit.HTTP.passthrough environment variable to true. Test the job by selecting Build now, and check that the job has built correctly. Now your tests run in a simulation mode on your development environment and in a passthrough mode on your CI server.

Every component family and component needs to have a representative icon. You have to define a custom icon as follows: For the component family the icon is defined in the package-info.java file. For the component itself, you need to declare the icon in the component class. Custom icons must comply with the following requirements: Icons must be stored in the src/main/resources/icons folder of the project. Icon file names need to match one of the following patterns: IconName.svg or IconName_icon32.png. The latter will run in degraded mode in Talend Cloud. Replace IconName by the name of your choice. Icons must be squared, even for the SVG format. Note that SVG icons are not supported by Talend Studio and can cause the deployment of the component to fail. If you aim at deploying a custom component to Talend Studio, specify PNG icons or use the Maven (or Gradle) svg2png plugin to convert SVG icons to PNG. If you want a finer control over both images, you can provide both in your component. Ultimately, you can also remove SVG parameters from the talend.component.server.icon.paths property in the HTTP server configuration. Note that SVG icons are not supported by Talend Studio and can cause the deployment of the component to fail. If you aim at deploying a custom component to Talend Studio, specify PNG icons or use the Maven (or Gradle) svg2png plugin to convert SVG icons to PNG. If you want a finer control over both images, you can provide both in your component. Ultimately, you can also remove SVG parameters from the talend.component.server.icon.paths property in the HTTP server configuration.

Before implementing a component logic and configuration, you need to specify the family and the category it belongs to, the component type and name, as well as its name and a few other generic parameters. This set of metadata, and more particularly the family, categories and component type, is mandatory to recognize and load the component to Talend Studio or Cloud applications. Some of these parameters are handled at the project generation using the starter, but can still be accessed and updated later on. The family and category of a component is automatically written in the package-info.java file of the component package, using the @Components annotation. By default, these parameters are already configured in this file when you import your project in your IDE. Their value correspond to what was defined during the project definition with the starter. Multiple components can share the same family and category value, but the family + name pair must be unique for the system. A component can belong to one family only and to one or several categories. If not specified, the category defaults to Misc. The package-info.java file also defines the component family icon, which is different from the component icon. You can learn how to customize this icon in this section. Here is a sample package-info.java: Another example with an existing component: Components can require metadata to be integrated in Talend Studio or Cloud platforms. Metadata is set on the component class and belongs to the org.talend.sdk.component.api.component package. When you generate your project and import it in your IDE, icon and version both come with a default value. @Icon: Sets an icon key used to represent the component. You can use a custom key with the custom() method but the icon may not be rendered properly. The icon defaults to Check. Replace it with a custom icon, as described in this section. @Version: Sets the component version. 1 by default. Learn how to manage different versions and migrations between your component versions in this section. For example: Every component family and component needs to have a representative icon. You have to define a custom icon as follows: For the component family the icon is defined in the package-info.java file. For the component itself, you need to declare the icon in the component class. Custom icons must comply with the following requirements: Icons must be stored in the src/main/resources/icons folder of the project. Icon file names need to match one of the following patterns: IconName.svg or IconName_icon32.png. The latter will run in degraded mode in Talend Cloud. Replace IconName by the name of your choice. Icons must be squared, even for the SVG format. Note that SVG icons are not supported by Talend Studio and can cause the deployment of the component to fail. If you aim at deploying a custom component to Talend Studio, specify PNG icons or use the Maven (or Gradle) svg2png plugin to convert SVG icons to PNG. If you want a finer control over both images, you can provide both in your component. Ultimately, you can also remove SVG parameters from the talend.component.server.icon.paths property in the HTTP server configuration. Note that SVG icons are not supported by Talend Studio and can cause the deployment of the component to fail. If you aim at deploying a custom component to Talend Studio, specify PNG icons or use the Maven (or Gradle) svg2png plugin to convert SVG icons to PNG. If you want a finer control over both images, you can provide both in your component. Ultimately, you can also remove SVG parameters from the talend.component.server.icon.paths property in the HTTP server configuration. For any purpose, you can also add user defined metadatas to your component with the @Metadatas annotation. Example: You can also use a SPI implementing org.talend.sdk.component.spi.component.ComponentMetadataEnricher. Methodology for creating components Generating a project using the starter Managing component versions Defining an input component Defining a processor or output component Defining a driver runner component Defining component layout and configuration Best practices

This document explains how Asciidoctor is used in the context of Talend Component Kit as well as the specific processes in place. For general guidelines about Asciidoctor, refer to the Asciidoc Syntax quick reference page. There are two ways to suggest modifications or new content. Both of the options below require you to have a GitHub account created. On every page of the Talend Component Kit Developer Guide, a Suggest and edit button is available. It allows you to access the corresponding source file on GitHub and to create a pull request with the suggested edits. The pull request is then assessed by the team in charge of the project. Fork the Runtime repository of the Talend Component Kit project and edit .adoc files located under documentation\src\main\antora\modules\ROOT\pages. Make sure to follow the guidelines outlined in the current document, especially for large modifications or new content, to make sure it can properly be rendered. When done, create a pull request that will be assessed by the team in charge of the project. The documentation is made of: Documentation files manually written under documentation\src\main\antora\modules\ROOT\pages. Documentation files automatically generated from the source code under documentation\src\main\antora\modules\ROOT\pages\_partials. These files are individually called in manually written files through includes. Assets, especially images, stored in the documentation\src\main\antora\modules\ROOT\assets folder. Some subfolders exist to categorize the assets. Each file has a unique name and is rendered as a unique HTML page. Some of the files are prefixed to help identifying the type of content it contains. Most common examples are: index- for pages referenced from the main index page. These pages also contain specific attributes to be correctly rendered on the main index page (see 'List of metadata attributes' below). tutorial- for tutorials/guided examples. generated_ for pages generated from the source code. These pages are generally stored in the _partials folder. For all pages: :page-partial indicates that the current .adoc file can be included in another document using an include::. This attribute has no value. :page-talend_skipindexation: indicates that the current .adoc file must not be indexed. This attribute has no value. Add it to files that should not be returned in the search, like index files that only contain includes. :description: is the meta description of the file. Each .adoc file is rendered as an HTML file. :keywords: is the list of meta keywords relevant for the current .adoc file. Separate keywords using simple commas. :page-talend_stage: draft indicates that the current document is a draft and is not final even if published. It triggers the display of a small banner indicating the status of the page. Remove this attribute once the page is final. For pages that should appear as a tile on the index page: :page-documentationindex-index: is the weight of the page. A low weight indicates that the page should be one of the first tiles to appear. A high weight will push the tile towards the end of the list in the index page. :page-documentationindex-label: is the title of the tile in the index page. :page-documentationindex-icon: is the icon of the tile in the index page. The value of this attribute should be the name of a free icon on fontawesome. :page-documentationindex-description: is a short description of the page that will be displayed in the tile under its title. For pages containing API descriptions: :page-talend_swaggerui: true indicates that the page contains some API reference that should be displayed using Swagger UI The Talend Component Kit documentation is published as HTML and PDF. Some parts can differ between these two versions, such as lists of links, that are not functional in the PDF version. To avoid this, it is possible to define some conditionality to have some content only displaying in one of the output formats only. For example: Every .adoc file can only contain one 'level 1' title (=). It is the title of the page and is located at the top of the document. It is a best practices that all sublevels added to the document are kept consistent. For example, don’t use a 'level 3' (===) section directly inside a 'level 1'. When possible, avoid going lower than section 2 (===) to keep the page readable. In the HTML output, the document title is rendered as

, section 1 titles as

, etc. The "in-page" navigation available on the right side of the HTML rendering only considers

and

elements. Other levels are ignored to keep the navigation readable and simple. It is possible to reuse content through "includes". Includes can be used to reuse entire files in other files, allowing to avoid copy pasting. When using an 'include' (calling an .adoc file from another .adoc file), you can specify a level offset to keep the hierarchy consistent in the current document. Avoid using includes if not absolutely necessary. An include can be done as follows: In this case, both doc1.adoc and doc2.adoc are rendered in the same page and their content is offset by one level, meaning that the document title of doc1 becomes a section 1 title (

) instead of an

in the final rendering, and so on. Note that both doc1.adoc and doc2.adoc will in addition be rendered as standalone pages (doc1.html and doc2.html). All images are stored under documentation > src > main > antora > modules > ROOT > assets > images. Relatively to .adoc files, it can be ../assets/images/ or ../../assets/images for _partials (automatically generated from code) pages. To avoid handling different relative paths, the backend already resolves directly image: to the image folder. Hence, paths to images should start with the following: image:(/).png[(,parameters)] If there is no subfolder, type the image name right away. Adding an image title is mandatory to avoid empty broken spaces in the content. If necessary, you can add more parameters separated by a comma between the same brackets as the image title, such as the desired width, height, etc. Use values in % for image size. For example; image:landscape.png[Landscape,70%,window="_blank",link="HTTPs://talend.github.io/component-runtime/main/1.38.16/_images/landscape.png"] In a general manner, avoid using tables if there are other solutions available. This is especially the case for complex tables that include assets or big code samples, as these can lead to display issues. Table example: value API Type Description @o.t.s.c.api.service.completion.DynamicValues dynamic_values Mark a method as being useful to fill potential values of a string option for a property denoted by its value. @o.t.s.c.api.service.healthcheck.HealthCheck healthcheck This class marks an action doing a connection test The following elements can be used to create admonition blocks. However, avoid using them one after another as it can make reading more difficult: NOTE: for a simple information note IMPORTANT: for a warning. Warnings should include information that lead to important errors if not taken into account TIP: for alternative ways or shortcuts to ease a process or procedure Admonition blocks should be kept as simple and short as possible. In some niche cases, it may be required to insert more complex content in an admonition block, such as a bullet list. In these cases, they should be formatted as follows:

Components built using Talend Component Kit can be shared as component archives (.car). These CAR files are executable files allowing to easily deploy the components it contains to any compatible version of Talend Studio. Component developers can generate .car files from their projects to share their components and make them available for other users, as detailed in this document. This document assumes that you have a component archive (.car) file and need to deploy it to Talend Studio. The component archive (.car) is executable and exposes the studio-deploy command which takes a Talend Studio home path as parameter. When executed, it installs the dependencies into the Studio and registers the component in your instance. For example: You can also upload the dependencies to your Nexus server using the following command: In this command, Nexus URL and repository name are mandatory arguments. All other arguments are optional. If arguments contain spaces or special symbols, you need to quote the whole value of the argument. For example: Talend Studio allows you to share components you have created using Talend Component Kit to other users working on the same remote project. Remote projects are available with Enterprise versions of Talend Studio only. Also, note that this feature has been removed in Studio since 7.3 release. Make sure you are connected to a remote project and the artifact repository for component sharing has been properly configured. On the toolbar of the Studio main window, click or click File > Edit Project Properties from the menu bar to open the Project Settings dialog box. In the tree view of the dialog box, select Repository Share to open the corresponding view. Select the Propagate components update to Artifact Repository check box. In the Repository ID field, specify the artifact repository configured for component sharing, and then click Check connection to verify the connectivity. Click Apply and Close to validate the settings and close the dialog box. Create a folder named patches at the root of your Talend Studio installation directory, then copy the .car files of the components you want share to this folder. Restart your Talend Studio and connect to the remote project. The components are deployed automatically to the repository and available in the Palette for other users when connected to a remote project with the same sharing repository configuration. My custom component builds correctly but does not appear in Talend Studio, how to fix it? This issue can be caused by the icon specified in the component metadata. Make sure to specify a custom icon for the component and the component family. These custom icons must be in PNG format to be properly handled by Talend Studio. Remove SVG parameters from the talend.component.server.icon.paths property in the HTTP server configuration. Refer to this section. Learn more about defining custom icons for components in this document.