Generating a project using the Component Kit Starter

The Component Kit Starter lets you design your components configuration and generates a ready-to-implement project structure.

The Starter is available on the web or as an IntelliJ plugin.

This tutorial shows you how to use the Component Kit Starter to generate new components for MySQL databases. Before starting, make sure that you have correctly setup your environment. See this section.

When defining a project using the Starter, do not refresh the page to avoid losing your configuration.

Configuring the project

Before being able to create components, you need to define the general settings of the project:

  1. Create a folder on your local machine to store the resource files of the component you want to create. For example, C:/my_components.

  2. Open the Starter in the web browser of your choice.

  3. Select your build tool. This tutorial uses Maven, but you can select Gradle instead.

  4. Add any facet you need. For example, add the Talend Component Kit Testing facet to your project to automatically generate unit tests for the components created in the project.

    Test facet

  5. Enter the Component Family of the components you want to develop in the project. This name must be a valid java name and is recommended to be capitalized, for example 'MySQL'.
    Once you have implemented your components in the Studio, this name is displayed in the Palette to group all of the MySQL-related components you develop, and is also part of your component name.

  6. Select the Category of the components you want to create in the current project. As MySQL is a kind of database, select Databases in this tutorial.
    This Databases category is used and displayed as the parent family of the MySQL group in the Palette of the Studio.

  7. Complete the project metadata by entering the Group, Artifact and Package.

  8. By default, you can only create processors. If you need to create Input or Output components, select Activate IO. By doing this:

    • Two new menu entries let you add datasets and datastores to your project, as they are required for input and output components.

      Input and Output components without dataset (itself containing a datastore) will not pass the validation step when building the components. Learn more about datasets and datastores in this document.

    • An Input component and an Output component are automatically added to your project and ready to be configured.

    • Components added to the project using Add A Component can now be processors, input or output components.

      Activate IO

Defining a Datastore

A datastore represents the data needed by an input or output component to connect to a database.

When building a component, the validateDataSet validation checks that each input or output (processor without output branch) component uses a dataset and that this dataset has a datastore.

You can define one or several datastores if you have selected the Activate IO step.

  1. Select Datastore. The list of datastores opens. By default, a datastore is already open but not configured. You can configure it or create a new one using Add new Datastore.

  2. Specify the name of the datastore. Modify the default value to a meaningful name for your project.
    This name must be a valid Java name as it will represent the datastore class in your project. It is a good practice to start it with an uppercase letter.

  3. Edit the datastore configuration. Parameter names must be valid Java names. Use lower case as much as possible. A typical configuration includes connection details to a database:

    • url

    • username

    • password.

  4. Save the datastore configuration.

Defining a Dataset

A dataset represents the data coming from or sent to a database and needed by input and output components to operate.

The validateDataSet validation checks that each input or output (processor without output branch) component uses a dataset and that this dataset has a datastore.

You can define one or several datasets if you have selected the Activate IO step.

  1. Select Dataset. The list of datasets opens. By default, a dataset is already open but not configured. You can configure it or create a new one using the Add new Dataset button.

  2. Specify the name of the dataset. Modify the default value to a meaningful name for your project.
    This name must be a valid Java name as it will represent the dataset class in your project. It is a good practice to start it with an uppercase letter.

  3. Edit the dataset configuration. Parameter names must be valid Java names. Use lower case as much as possible. A typical configuration includes details of the data to retrieve:

    • Datastore to use (that contains the connection details to the database)

    • table name

    • data

  4. Save the dataset configuration.

Creating an Input component

To create an input component, make sure you have selected Activate IO.

When clicking Add A Component in the Starter, a new step allows you to define a new component in your project.
The intent in this tutorial is to create an input component that connects to a MySQL database, executes a SQL query and gets the result.

Input configuration

  1. Choose the component type. Input in this case.

  2. Enter the component name. For example, MySQLInput.

  3. Click Configuration model. This button lets you specify the required configuration for the component. By default, a dataset is already specified.

  4. For each parameter that you need to add, click the (+) button on the right panel. Enter the parameter name and choose its type then click the tick button to save the changes.
    In this tutorial, to be able to execute a SQL query on the Input MySQL database, the configuration requires the following parameters:+

    • a dataset (which contains the datastore with the connection information)

    • a timeout parameter.

      Closing the configuration panel on the right does not delete your configuration. However, refreshing the page resets the configuration.
      Input configuration

  5. Specify whether the component issues a stream or not. In this tutorial, the MySQL input component created is an ordinary (non streaming) component. In this case, leave the Stream option disabled.

  6. Select the Record Type generated by the component. In this tutorial, select Generic because the component is designed to generate records in the default Record format.
    You can also select Custom to define a POJO that represents your records.

Your input component is now defined. You can add another component or generate and download your project.

Creating a Processor component

When clicking Add A Component in the Starter, a new step allows you to define a new component in your project. The intent in this tutorial is to create a simple processor component that receives a record, logs it and returns it at it is.

If you did not select Activate IO, all new components you add to the project are processors by default.
If you selected Activate IO, you can choose the component type. In this case, to create a Processor component, you have to manually add at least one output.

  1. If required, choose the component type: Processor in this case.

  2. Enter the component name. For example, RecordLogger, as the processor created in this tutorial logs the records.

  3. Specify the Configuration Model of the component. In this tutorial, the component doesn’t need any specific configuration. Skip this step.

  4. Define the Input(s) of the component. For each input that you need to define, click Add Input. In this tutorial, only one input is needed to receive the record to log.

  5. Click the input name to access its configuration. You can change the name of the input and define its structure using a POJO. If you added several inputs, repeat this step for each one of them.
    The input in this tutorial is a generic record. Enable the Generic option and click Save.

    Processor configuration

  6. Define the Output(s) of the component. For each output that you need to define, click Add Output. The first output must be named MAIN. In this tutorial, only one generic output is needed to return the received record.
    Outputs can be configured the same way as inputs (see previous steps).
    You can define a reject output connection by naming it REJECT. This naming is used by Talend applications to automatically set the connection type to Reject.

Your processor component is now defined. You can add another component or generate and download your project.

Creating an Output component

To create an output component, make sure you have selected Activate IO.

When clicking Add A Component in the Starter, a new step allows you to define a new component in your project.
The intent in this tutorial is to create an output component that receives a record and inserts it into a MySQL database table.

Output components are Processors without any output. In other words, the output is a processor that does not produce any records.
Output configuration

  1. Choose the component type. Output in this case.

  2. Enter the component name. For example, MySQLOutput.

  3. Click Configuration Model. This button lets you specify the required configuration for the component. By default, a dataset is already specified.

  4. For each parameter that you need to add, click the (+) button on the right panel. Enter the name and choose the type of the parameter, then click the tick button to save the changes.
    In this tutorial, to be able to insert a record in the output MySQL database, the configuration requires the following parameters:+

    • a dataset (which contains the datastore with the connection information)

    • a timeout parameter.

      Closing the configuration panel on the right does not delete your configuration. However, refreshing the page resets the configuration.

  5. Define the Input(s) of the component. For each input that you need to define, click Add Input. In this tutorial, only one input is needed.

  6. Click the input name to access its configuration. You can change the name of the input and define its structure using a POJO. If you added several inputs, repeat this step for each one of them.
    The input in this tutorial is a generic record. Enable the Generic option and click Save.

Do not create any output because the component does not produce any record. This is the only difference between an output an a processor component.

Your output component is now defined. You can add another component or generate and download your project.

Generating and downloading the final project

Once your project is configured and all the components you need are created, you can generate and download the final project. In this tutorial, the project was configured and three components of different types (input, processor and output) have been defined.

  1. Click Finish on the left panel. You are redirected to a page that summarizes the project. On the left panel, you can also see all the components that you added to the project.

    Output configuration

  2. Generate the project using one of the two options available:

    • Download it locally as a ZIP file using the Download as ZIP button.

    • Create a GitHub repository and push the project to it using the Create on Github button.

In this tutorial, the project is downloaded to the local machine as a ZIP file.

Compiling and exploring the generated project files

Once the package is available on your machine, you can compile it using the build tool selected when configuring the project.

  • In the tutorial, Maven is the build tool selected for the project.
    In the project directory, execute the mvn package command.
    If you don’t have Maven installed on your machine, you can use the Maven wrapper provided in the generated project, by executing the ./mvnw package command.

  • If you have created a Gradle project, you can compile it using the gradle build command or using the Gradle wrapper: ./gradlew build.

The generated project code contains documentation that can guide and help you implementing the component logic. Import the project to your favorite IDE to start the implementation.

Generating a project using an OpenAPI JSON descriptor

The Component Kit Starter allows you to generate a component development project from an OpenAPI JSON descriptor.

  1. Open the Starter in the web browser of your choice.

  2. Enable the OpenAPI mode using the toggle in the header.

    Open API toggle

  3. Go to the API menu.

  4. Paste the OpenAPI JSON descriptor in the right part of the screen. All the described endpoints are detected.

  5. Unselect the endpoints that you do not want to use in the future components. By default, all detected endpoints are selected.

    Open API menu

  6. Go to the Finish menu.

  7. Download the project.

When exploring the project generated from an OpenAPI descriptor, you can notice the following elements:

  • sources

  • the API dataset

  • an HTTP client for the API

  • a connection folder containing the component configuration. By default, the configuration is only made of a simple datastore with a baseUrl parameter.

Related articles

Scroll to top