Search results for overview

Implementing components  Get an overview of the main steps to code the logic of your custom Talend Componit Kit components   create code class logic layout configuration dev overview api

Once you have generated a project, you can start implementing the logic and layout of your components and iterate on it. Depending on the type of component you want to create, the logic implementation can differ. However, the layout and component metadata are defined the same way for all types of components in your project. The main steps are: Defining family and component metadata Defining an input component logic Defining a processor/output logic Defining a standalone component logic Defining component layout and configuration In some cases, you will require specific implementations to handle more advanced cases, such as: Internationalizing a component Managing component versions Masking sensitive data Implementing batch processing Implementing streaming on a component You can also make certain configurations reusable across your project by defining services. Using your Java IDE along with a build tool supported by the framework, you can then compile your components to test and deploy them to Talend Studio or other Talend applications: Building components with Maven Building components with Gradle Wrapping a Beam I/O In any case, follow these best practices to ensure the components you develop are optimized. You can also learn more about component loading and plugins here: Loading a component

Defining services  Get an overview of what are services and how to use services with Talend Component Kit   service component-manager reuse configuration overview action custom interceptor i18n api

Services are configurations that can be reused across several classes. Talend Component Kit comes with a predefined set of services that you can easily use. You can still define your own services under the service node of your component project. By default, the Component Kit Starter generates a dedicated class in your project in which you can implement services. Built-in services Internationalizing a service Providing actions through a service Services and interceptors Defining a custom API

Generating a project  Get an overview of the first step to create a new component, which consists in building a project using the Component Kit Starter or the Component Kit plugin for IntelliJ   starter starter overview component project intelliJ intellij

The first step when developing new components is to create a project that will contain the skeleton of your components and set you on the right track. The project generation can be achieved using the Talend Component Kit Starter or the Talend Component Kit plugin for IntelliJ. Through a user-friendly interface, you can define the main lines of your project and of your component(s), including their name, family, type, configuration model, and so on. Once completed, all the information filled are used to generate a project that you will use as starting point to implement the logic and layout of your components, and to iterate on them. Using the starter Using the IntelliJ plugin Once your project is generated, you can start implementing the component logic.

Getting started with Talend Component Kit  Learn the basics about Talend Component Kit framework and get ready to create new components   quickstart overview principle description

Talend Component Kit is a Java framework designed to simplify the development of components at two levels: The Runtime, that injects the specific component code into a job or pipeline. The framework helps unifying as much as possible the code required to run in Data Integration (DI) and BEAM environments. The Graphical interface. The framework helps unifying the code required to render the component in a browser or in the Eclipse-based Talend Studio (SWT). Most part of the development happens as a Maven or Gradle project and requires a dedicated tool such as IntelliJ. The Component Kit is made of: A Starter, that is a graphical interface allowing you to define the skeleton of your development project. APIs to implement components UI and runtime. Development tools: Maven and Gradle wrappers, validation rules, packaging, Web preview, etc. A testing kit based on JUnit 4 and 5. By using this tooling in a development environment, you can start creating components as described below. Developing new components using the Component Kit framework includes: Creating a project using the starter or the Talend IntelliJ plugin. This step allows to build the skeleton of the project. It consists in: Defining the general configuration model for each component in your project. Generating and downloading the project archive from the starter. Compiling the project. Importing the compiled project in your IDE. This step is not required if you have generated the project using the IntelliJ plugin. Implementing the components, including: Registering the components by specifying their metadata: family, categories, version, icon, type and name. Defining the layout and configurable part of the components. Defining the execution logic of the components, also called runtime. Testing the components. Deploying the components to Talend Studio or Cloud applications. Optionally, you can use services. Services are predefined or user-defined configurations that can be reused in several components. There are four types of components, each type coming with its specificities, especially on the runtime side. Input components: Retrieve the data to process from a defined source. An input component is made of: The execution logic of the component, represented by a Mapper or an Emitter class. The source logic of the component, represented by a Source class. The layout of the component and the configuration that the end-user will need to provide when using the component, defined by a Configuration class. All input components must have a dataset specified in their configuration, and every dataset must use a datastore. Processors: Process and transform the data. A processor is made of: The execution logic of the component, describing how to process each records or batches of records it receives. It also describes how to pass records to its output connections. This logic is defined in a Processor class. The layout of the component and the configuration that the end-user will need to provide when using the component, defined by a Configuration class. Output components: Send the processed data to a defined destination. An output component is made of: The execution logic of the component, describing how to process each records or batches of records it receives. This logic is defined in an Output class. Unlike processors, output components are the last components of the execution and return no data. The layout of the component and the configuration that the end-user will need to provide when using the component, defined by a Configuration class. All input components must have a dataset specified in their configuration, and every dataset must use a datastore. Standalone components: Make a call to the service or run a query on the database. A standalone component is made of: The execution logic of the component, represented by a DriverRunner class. The layout of the component and the configuration that the end-user will need to provide when using the component, defined by a Configuration class. All input components must have a datastore or dataset specified in their configuration, and every dataset must use a datastore. The following example shows the different classes of an input components in a multi-component development project: Setup your development environment Generate your first project and develop your first component

Testing components  Learn how to test your component logic in the environment you need using Talend Component Kit   test overview environment beam runtime testing

Developing new components includes testing them in the required execution environments. Use the following articles to learn about the best practices and the available options to fully test your components. Component testing best practices Component testing kit Beam testing Testing in multiple environments Reusing Maven credentials Generating data for testing Simple/Test Pipeline API Beam Pipeline API

Integrating components into Talend Studio  Get an overview of how to integrate your components created using Talend Component Kit to Talend Studio   integration deployment open studio studio studio-integration

To be able to see and use your newly developed components, you need to integrate them to the right application. Currently, you can deploy your components to Talend Studio as part of your development process to iterate on them: Iterating on component development with Talend Studio You can also share your components externally and install them using a component archive (.car) file. Sharing and installing components in Talend Studio Check the versions of the framework that are compatible with your version of Talend Studio in this document. If you were used to create custom components with the Javajet framework and want to get to know the new approach and main differences of the Component Kit framework, refer to this document.

Talend Component Kit Overview  Learn the basic concepts of the Talend Component Kit framework   framework

Talend Component Kit is a toolkit based on Java and designed to simplify the development of components at two levels: Runtime: Runtime is about injecting the specific component code into a job or pipeline. The framework helps unify as much as possible the code required to run in Data Integration (DI) and BEAM environments. Graphical interface: The framework helps unify the code required to be able to render the component in a browser (web) or in the Eclipse-based Studio (SWT). The Talend Component Kit framework is made of several tools designed to help you during the component development process. It allows to develop components that fit in both Java web UIs. Starter: Generate the skeleton of your development project using a user-friendly interface. The Talend Component Kit Starter is available as a web tool or as a plugin for the IntelliJ IDE. Component API: Check all classes available to implement components. Build tools: The framework comes with Maven and Gradle wrappers, which allow to always use the version of Maven or Gradle that is right for your component development environment and version. Testing tools: Test components before integrating them into Talend Studio or Cloud applications. Testing tools include the Talend Component Kit Web Tester, which allows to check the web UI of your components on your local machine. You can find more details about the framework design in this document. The Talend Component Kit project is available on GitHub in the following repository

Talend Component Kit design overview    

The component API is declarative (through annotations) to ensure it is: Evolutive. It can get new features without breaking old code. As static as possible. Because it is fully declarative, any new API can be added iteratively without requiring any change to existing components. For example, in the case of Beam potential evolution: would not be affected by the addition of the new Timer API, which can be used as follows: The intent of the framework is to be able to fit in a Java UI as well as in a web UI. It must be understood as colocalized and remote UI. The goal is to move as much as possible the logic to the UI side for UI-related actions. For example, validating a pattern, a size, and so on, should be done on the client side rather than on the server side. Being static encourages this practice. The other goal of being static in the API definition is to ensure that the model will not be mutated at runtime and that all the auditing and modeling can be done before, at the design phase. Being static also ensures that the development can be validated as much as possible through build tools. This does not replace the requirement to test the components but helps developers to maintain components with automated tools. Refer to this document. The components must be able to execute even if they have conflicting libraries. For that purpose, classloaders must be isolated. A component defines its dependencies based on a Maven format and is always bound to its own classloader. The definition payload is as flat as possible and strongly typed to ensure it can be manipulated by consumers. This way, consumers can add or remove fields with simple mapping rules, without any abstract tree handling. The execution (runtime) configuration is the concatenation of framework metadata (only the version) and a key/value model of the instance of the configuration based on the definition properties paths for the keys. It enables consumers to maintain and work with the keys/values according to their need. The framework not being responsible for any persistence, it is very important to make sure that consumers can handle it from end to end, with the ability to search for values (update a machine, update a port and so on) and keys (for example, a new encryption rule on key certificate). Talend Component Kit is a metamodel provider (to build forms) and a runtime execution platform. It takes a configuration instance and uses it volatilely to execute a component logic. This implies it cannot own the data nor define the contract it has for these two endpoints and must let the consumers handle the data lifecycle (creation, encryption, deletion, and so on). A new mime type called talend/stream is introduced to define a streaming format. It matches a JSON object per line: Icons (@Icon) are based on a fixed set. Custom icons can be used but their display cannot be guaranteed. Components can be used in any environment and require a consistent look that cannot be guaranteed outside of the UI itself. Defining keys only is the best way to communicate this information. Once you know exactly how you will deploy your component in the Studio, then you can use `@Icon(value = CUSTOM, custom = "…") to use a custom icon file.