Testing on multiple environments

JUnit (4 or 5) already provides ways to parameterize tests and execute the same "test logic" against several sets of data. However, it is not very convenient for testing multiple environments.

For example, with Beam, you can test your code against multiple runners. But it requires resolving conflicts between runner dependencies, setting the correct classloaders, and so on.

To simplify such cases, the framework provides you a multi-environment support for your tests, through the JUnit module, which works with both JUnit 4 and JUnit 5.

JUnit 4

@RunWith(MultiEnvironmentsRunner.class)
@Environment(Env1.class)
@Environment(Env2.class)
public class TheTest {
    @Test
    public void test1() {
        // ...
    }
}

The MultiEnvironmentsRunner executes the tests for each defined environments. With the example above, it means that it runs test1 for Env1 and Env2.

By default, the JUnit4 runner is used to execute the tests in one environment, but you can use @DelegateRunWith to use another runner.

JUnit 5

The multi-environment configuration with JUnit 5 is similar to JUnit 4:

@Environment(EnvironmentsExtensionTest.E1.class)
@Environment(EnvironmentsExtensionTest.E2.class)
class TheTest {

    @EnvironmentalTest
    void test1() {
        // ...
    }
}

The main differences are that no runner is used because they do not exist in JUnit 5, and that you need to replace @Test by @EnvironmentalTest.

With JUnit5, tests are executed one after another for all environments, while tests are ran sequentially in each environments with JUnit 4. For example, this means that @BeforeAll and @AfterAll are executed once for all runners.

Provided environments

The provided environment sets the contextual classloader in order to load the related runner of Apache Beam.

Package: org.talend.sdk.component.junit.environment.builtin.beam

the configuration is read from system properties, environment variables, …​.
Contextual

_class: ContextualEnvironment.

Direct

_class: DirectRunnerEnvironment.

Flink

_class: FlinkRunnerEnvironment.

Spark

_class: SparkRunnerEnvironment.

Configuring environments

If the environment extends BaseEnvironmentProvider and therefore defines an environment name - which is the case of the default ones - you can use EnvironmentConfiguration to customize the system properties used for that environment:

@Environment(DirectRunnerEnvironment.class)
@EnvironmentConfiguration(
    environment = "Direct",
    systemProperties = @EnvironmentConfiguration.Property(key = "beamTestPipelineOptions", value = "..."))

@Environment(SparkRunnerEnvironment.class)
@EnvironmentConfiguration(
    environment = "Spark",
    systemProperties = @EnvironmentConfiguration.Property(key = "beamTestPipelineOptions", value = "..."))

@Environment(FlinkRunnerEnvironment.class)
@EnvironmentConfiguration(
    environment = "Flink",
    systemProperties = @EnvironmentConfiguration.Property(key = "beamTestPipelineOptions", value = "..."))
class MyBeamTest {

    @EnvironmentalTest
    void execute() {
        // run some pipeline
    }
}
If you set the <environment name>.skip system property to true, the environment-related executions are skipped.

Advanced usage

This usage assumes that Beam 2.4.0 or later is used.

The following dependencies bring the JUnit testing toolkit, the Beam integration and the multi-environment testing toolkit for JUnit into the test scope.

Dependencies:

<dependencies>
  <dependency>
    <groupId>org.talend.sdk.component</groupId>
    <artifactId>component-runtime-junit</artifactId>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-api</artifactId>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.jboss.shrinkwrap.resolver</groupId>
    <artifactId>shrinkwrap-resolver-impl-maven</artifactId>
    <version>3.1.4</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.talend.sdk.component</groupId>
    <artifactId>component-runtime-beam</artifactId>
    <scope>test</scope>
  </dependency>
</dependencies>

Using the fluent DSL to define jobs, you can write a test as follows:

Your job must be linear and each step must send a single value (no multi-input or multi-output). 
@Environment(ContextualEnvironment.class)
@Environment(DirectRunnerEnvironment.class)
class TheComponentTest {
    @EnvironmentalTest
    void testWithStandaloneAndBeamEnvironments() {
        from("myfamily://in?config=xxxx")
            .to("myfamily://out")
            .create()
            .execute();
        // add asserts on the output if needed
    }
}

It executes the chain twice:

  1. With a standalone environment to simulate the Studio.

  2. With a Beam (direct runner) environment to ensure the portability of your job.

Related articles

Scroll to top