Chaining patterns together

In a chain of compilers, any output layer of a given compiler may be used as additional input by any compiler down the chain.

Usually the tasks that the Driver executes contain the implementation of one single pattern. However, you can also use the processing library to create tasks that comprise of multiple chained compilation patterns. This way, you can reduce the number of Pipelines to define for a particular processing task in a Pipeline Topology.

Limitations

Different compilers cannot write to the same output layer. The processing library checks whether this condition is met and raises an exception if two compilers are trying to write to the same output layer.

If a given compiler needs one or more output layer as input, these layers must have been produced by one of the compiler present in the chain before the given one. This condition is also checked by the library and an exception is raised if it occurs.

Building a multi-compiler task

The example below illustrates this task and its related builder with a multi-compiler task built out of two different types of compilers. Since not all patterns support incremental compilation, the resulting task always reprocesses the whole input, ignoring the changes. This is not true when all the chained compilers support incremental compilation, in this case the whole chain runs incrementally when incremental processing is applicable.

def configureCompiler(completeConfig: CompleteConfig,
                      context: DriverContext,
                      builder: DriverBuilder): builder.type = {

  // The task Id
  val taskId = "Id"

  // Create the two compilers according to limitations of multi-execution
  val myNonIncrementalCompiler = new MyNonIncrementalCompiler(context)
  val myDepCompiler = new MyDepCompiler(context)

  // Create the multi-compiler builder
  val multiCompilerTaskBuilder =
    builder
      .newMultiCompilerTaskBuilder(taskId)
      .addNonIncrementalCompiler(myNonIncrementalCompiler)
      .addDepCompiler(myDepCompiler)

  // Build the multi-compiler driver task and add it to the driver builder
  val multiCompilerTask = multiCompilerTaskBuilder.build()
  builder.addTask(multiCompilerTask)
}