component-runtime-junit

component-runtime-junit is a test library that allows you to validate simple logic based on the Talend Component Kit tooling.

To import it, add the following dependency to your project:

<dependency>
  <groupId>org.talend.sdk.component</groupId>
  <artifactId>component-runtime-junit</artifactId>
  <version>${talend-component.version}</version>
  <scope>test</scope>
</dependency>

This dependency also provides mocked components that you can use with your own component to create tests.

The mocked components are provided under the test family:

  • emitter : a mock of an input component

  • collector : a mock of an output component

The collector is "per thread" by default. If you are executing a Beam (or concurrent) job, it will not work. To switch to a JVM wide storage, set the talend.component.junit.handler.state system property to static (default being thread). You can do it in a maven-surefire-plugin execution.

JUnit 4

You can define a standard JUnit test and use the SimpleComponentRule rule:

public class MyComponentTest {

    @Rule (1)
    public final SimpleComponentRule components = new SimpleComponentRule("org.talend.sdk.component.mycomponent");

    @Test
    public void produce() {
        Job.components() (2)
             .component("mycomponent","yourcomponentfamily://yourcomponent?"+createComponentConfig())
             .component("collector", "test://collector")
           .connections()
             .from("mycomponent").to("collector")
           .build()
           .run();

        final List<MyRecord> records = components.getCollectedData(MyRecord.class); (3)
        doAssertRecords(records); // depending your test
    }
}
1 The rule creates a component manager and provides two mock components: an emitter and a collector. Set the root package of your component to enable it.
2 Define any chain that you want to test. It generally uses the mock as source or collector.
3 Validate your component behavior. For a source, you can assert that the right records were emitted in the mock collect.
The rule can also be defined as a @ClassRule to start it once per class and not per test as with @Rule.

To go further, you can add the ServiceInjectionRule rule, which allows to inject all the component family services into the test class by marking test class fields with @Service:

public class SimpleComponentRuleTest {

    @ClassRule
    public static final SimpleComponentRule COMPONENT_FACTORY = new SimpleComponentRule("...");

    @Rule (1)
    public final ServiceInjectionRule injections = new ServiceInjectionRule(COMPONENT_FACTORY, this); (2)

    @Service (3)
    private LocalConfiguration configuration;

    @Service
    private Jsonb jsonb;

    @Test
    public void test() {
        // ...
    }
}
1 The injection requires the test instance, so it must be a @Rule rather than a @ClassRule.
2 The ComponentsController is passed to the rule, which for JUnit 4 is the SimpleComponentRule, as well as the test instance to inject services in.
3 All service fields are marked with @Service to let the rule inject them before the test is ran.

JUnit 5

The JUnit 5 integration is very similar to JUnit 4, except that it uses the JUnit 5 extension mechanism.

The entry point is the @WithComponents annotation that you add to your test class, and which takes the component package you want to test. You can use @Injected to inject an instance of ComponentsHandler - which exposes the same utilities than the JUnit 4 rule - in a test class field :

@WithComponents("org.talend.sdk.component.junit.component") (1)
public class ComponentExtensionTest {
    @Injected (2)
    private ComponentsHandler handler;

    @Test
    public void manualMapper() {
        final Mapper mapper = handler.createMapper(Source.class, new Source.Config() {

            {
                values = asList("a", "b");
            }
        });
        assertFalse(mapper.isStream());
        final Input input = mapper.create();
        assertEquals("a", input.next());
        assertEquals("b", input.next());
        assertNull(input.next());
    }
}
1 The annotation defines which components to register in the test context.
2 The field allows to get the handler to be able to orchestrate the tests.
If you use JUnit 5 for the first time, keep in mind that the imports changed and that you need to use org.junit.jupiter.api.Test instead of org.junit.Test. Some IDE versions and surefire versions can also require you to install either a plugin or a specific configuration.

As for JUnit 4, you can go further by injecting test class fields marked with @Service, but there is no additional extension to specify in this case:

@WithComponents("...")
class ComponentExtensionTest {

    @Service (1)
    private LocalConfiguration configuration;

    @Service
    private Jsonb jsonb;

    @Test
    void test() {
        // ...
    }
}
1 All service fields are marked with @Service to let the rule inject them before the test is ran.

Streaming components

Streaming components have the issue to not stop by design. The Job DSL exposes two properties to help with that issue:

  • streaming.maxRecords: enables to request a maximum number of records

  • streaming.maxDurationMs: enables to request a maximum duration for the execution of the input

You can set them as properties on the job:

job.property("streaming.maxRecords", 5);

Mocking the output

Using the test://collector component as shown in the previous sample stores all records emitted by the chain (typically your source) in memory. You can then access them using theSimpleComponentRule.getCollectedData(type).

Note that this method filters by type. If you don’t need any specific type, you can use Object.class.

Mocking the input

The input mocking is symmetric to the output. In this case, you provide the data you want to inject:

public class MyComponentTest {

    @Rule
    public final SimpleComponentRule components = new SimpleComponentRule("org.talend.sdk.component.mycomponent");

    @Test
    public void produce() {
        components.setInputData(asList(createData(), createData(), createData())); (1)

        Job.components()
             .component("emitter","test://emitter")
             .component("out", "yourcomponentfamily://myoutput?"+createComponentConfig())
           .connections()
              .from("emitter").to("out")
           .build()
           .run();

        assertMyOutputProcessedTheInputData();
    }
}
1 using setInputData, you prepare the execution(s) to have a fake input when using the "test"/"emitter" component.

Creating runtime configuration from component configuration

The component configuration is a POJO (using @Option on fields) and the runtime configuration (ExecutionChainBuilder) uses a Map<String, String>. To make the conversion easier, the JUnit integration provides a SimpleFactory.configurationByExample utility to get this map instance from a configuration instance.

Example:

final MyComponentConfig componentConfig = new MyComponentConfig();
componentConfig.setUser("....");
// .. other inits

final Map<String, String> configuration = configurationByExample(componentConfig);

The same factory provides a fluent DSL to create the configuration by calling configurationByExample without any parameter. The advantage is to be able to convert an object as a Map<String, String> or as a query string in order to use it with the Job DSL:

final String uri = "family://component?" +
    configurationByExample().forInstance(componentConfig).configured().toQueryString();

It handles the encoding of the URI to ensure it is correctly done.

When writing tests for your components, you can force the maxBatchSize parameter value by setting it with the following syntax: $configuration.$maxBatchSize=10.

Testing a Mapper

The SimpleComponentRule also allows to test a mapper unitarily. You can get an instance from a configuration and execute this instance to collect the output.

Example:

public class MapperTest {

    @ClassRule
    public static final SimpleComponentRule COMPONENT_FACTORY = new SimpleComponentRule(
            "org.company.talend.component");

    @Test
    public void mapper() {
        final Mapper mapper = COMPONENT_FACTORY.createMapper(MyMapper.class, new Source.Config() {{
            values = asList("a", "b");
        }});
        assertEquals(asList("a", "b"), COMPONENT_FACTORY.collectAsList(String.class, mapper));
    }
}

Testing a Processor

As for a mapper, a processor is testable unitary. However, this case can be more complex in case of multiple inputs or outputs.

Example:

public class ProcessorTest {

    @ClassRule
    public static final SimpleComponentRule COMPONENT_FACTORY = new SimpleComponentRule(
            "org.company.talend.component");

    @Test
    public void processor() {
        final Processor processor = COMPONENT_FACTORY.createProcessor(Transform.class, null);
        final SimpleComponentRule.Outputs outputs = COMPONENT_FACTORY.collect(processor,
                        new JoinInputFactory().withInput("__default__", asList(new Transform.Record("a"), new Transform.Record("bb")))
                                              .withInput("second", asList(new Transform.Record("1"), new Transform.Record("2")))
                );
        assertEquals(2, outputs.size());
        assertEquals(asList(2, 3), outputs.get(Integer.class, "size"));
        assertEquals(asList("a1", "bb2"), outputs.get(String.class, "value"));
    }
}

The rule allows you to instantiate a Processor from your code, and then to collect the output from the inputs you pass in. There are two convenient implementations of the input factory:

  1. MainInputFactory for processors using only the default input.

  2. JoinInputfactory with the withInput(branch, data) method for processors using multiple inputs. The first argument is the branch name and the second argument is the data used by the branch.

If needed, you can also implement your own input representation using org.talend.sdk.component.junit.ControllableInputFactory.
Scroll to top