Using Camunda as an orchestrater in micro-services

1. What is Microservice Orchestration?

Microservices are an architectural style of building applications where each service is a separate codebase, which can be managed by a small team of developers independently of the other services. Microservices are designed to be highly scalable and fault-tolerant, allowing applications to be built more quickly and with better reliability.

Each service is designed to be as modular as possible and work independently, but they complete a business scope together. Choreography and orchestration are two approaches to communication between microservices. In microservices choreography, each service is responsible for its own communication with other services. In contrast, in microservices orchestration, a single service acts as a controller (or orchestrator) and directs the communication between services.

Microservice Orchestration involves managing the dependencies between services, ensuring that they are running and communicating correctly, and managing the flow of data between them. Orchestration can be done manually or with the help of automated orchestration platforms such as Camunda and Netflix Conductor, which allow developers to create complex workflows without having to manually manage each service.

You can imagine Microservice Orchestration as an orchestra where a central conductor is responsible for keeping the orchestra in sync and coordinating the members to produce a cohesive musical piece.

2. What is Camunda and how it works?

Before talking about Camunda, you need to understand BPMN first. Business Process Model and Notation (BPMN) is the global standard for process modeling. Below is a very simple BPMN model for the KYC process which verifies a newly created account before onboarding. BPMN is just a model like ERD, you can not use it to develop an application that why we need Camunda.

Camunda is an open-source workflow and decision automation platform. It enables users to model, automate, and optimize business processes, turning them into executable workflows using BPMN. Camunda is designed to be a highly scalable and extensible platform that can be used to create complex workflows and processes.

Camunda provides a set of REST APIs that can be used to interact with the engine programmatically. This means that you can use any programming language to communicate with the Camunda engine and manage processes and tasks.

3. Microservices orchestration with Camunda

We will build a demo project by Spring Boot which simulates a microservices system to help you understand how you can start your microservice orchestration journey with Camunda. So being familar with Spring Boot is a prerequisite to understand this demo.

This project is also a KYC system but more complicated than the example I showed you above. The system includes 4 services communicating via REST API:

Request Service: manage requests including storing new requests, approving and rejecting the request
Account Service: contains some features for a customer after it is verified successfully by the KYC process such as onboard new customer
Verification Service: contains the main logic of the KYC process
Orchestrater Service: orchestrate all services in the system

The BPMN model of this system is as below. When a customer creates a new account, the account name will be sent to our system, then we will create a request by Request Service then the request will be verified by Verification Service. If the verification is successful, this request will be approved and onboard this newly created customer by Account Service in parallel (thank for Camunda), otherwise the request will be rejected.

3.1. Tech stack

Java 11
Gradle
Spring boot
Spring Feign
Spring Camunda Web app

plugins {
	id 'org.springframework.boot' version '2.3.0.RELEASE'
	id 'io.spring.dependency-management' version '1.0.9.RELEASE'
	id 'java'
	id 'io.freefair.lombok' version '5.0.0'
}

group = 'guru.springframework'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '11'

repositories {
	mavenCentral()
}

ext {
	set('springCloudVersion', 'Hoxton.SR5')
}

dependencies {
	implementation 'org.springframework.boot:spring-boot-starter-web'
	testImplementation 'org.springframework.boot:spring-boot-starter-test'
	implementation 'org.camunda.bpm.springboot:camunda-bpm-spring-boot-starter-webapp:7.16.0'
	implementation 'com.h2database:h2:2.1.214'
	implementation 'org.springframework.boot:spring-boot-starter-jdbc'
	implementation('org.springframework.cloud:spring-cloud-starter-openfeign')
}

dependencyManagement {
	imports {
		mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
	}
}

test {
	useJUnitPlatform()
}

3.2. Simulate service’s APIs

Each step (or task) in the KYC process is implemented by a REST API accordingly. So we create these APIs using Spring RestController. On each API we will just a simple line of log to know if it is triggered or not.

@RestController
public class AccountServiceEndpoint {

  private static final Logger LOGGER = LoggerFactory.getLogger(AccountServiceEndpoint.class);

  @PostMapping("/account/onboard/{userName}")
  public void onboard(@PathVariable String userName){
    LOGGER.info("Onboard account {}", userName);
  }

  @PostMapping("/account/reject/{userName}")
  public void reject(@PathVariable String userName) throws InterruptedException {
    LOGGER.info("Reject account {}", userName);
  }
}

@RestController
public class RequestManagementServiceEndpoint {

  private static final Logger LOGGER = LoggerFactory.getLogger(RequestManagementServiceEndpoint.class);

  @PostMapping("/request/{userName}")
  public void storeRequest(@PathVariable String userName){
    LOGGER.info("Store request user name {}", userName);
  }

  @PostMapping("/request/approve/{userName}")
  public void approveRequest(@PathVariable String userName){
    LOGGER.info("Approve request user name {}", userName);
  }

  @PostMapping("/request/reject/{userName}")
  public void rejectRequest(@PathVariable String userName){
    LOGGER.info("Reject request user name {}", userName);
  }
}

@RestController
public class VerificationServiceEndpoint {

  private static final Logger LOGGER = LoggerFactory.getLogger(VerificationServiceEndpoint.class);

  @PostMapping("/verify/{userName}")
  public ResponseEntity verify(@PathVariable String userName) throws InterruptedException {
    if (userName.equals("thoainguyen")) {
      LOGGER.info("Verify success user name {}", userName);
      return ResponseEntity.ok().build();
    }
    else {
      LOGGER.info("Verify error user name {}", userName);
      return ResponseEntity.badRequest().build();
    }
  }
}

3.3. Camunda integration

In order to enable Camunda, we need to add the Spring Camunda Web app dependency in grable.build which will automatically include the Camunda engine and web apps. Then enable it in Application.java

@SpringBootApplication
@EnableFeignClients(basePackages = "com.thoainguyen.client")
@EnableProcessApplication
public class Application {

	public static void main(String[] args) {
		SpringApplication.run(Application.class, args);
	}

	@Bean
	public DataSource h2DataSource() {
		EmbeddedDatabase embeddedDatabase = new EmbeddedDatabaseBuilder()
			.setType(EmbeddedDatabaseType.H2)
			.setName(UUID.randomUUID() + ";Mode=Oracle;DEFAULT_NULL_ORDERING=HIGH")
			.build();
		Mode mode = Mode.getInstance("ORACLE");
		mode.limit = true;
		// Here is the trick
		mode.numericWithBooleanComparison = true;
		return embeddedDatabase;
	}
}

Camunda requires a data store to store the process information, in this demo, I will use H2 an in-memory database but in reality, we normally use RDBMS like PorgresDB or MySQL.

We also need to have META-INF/processes.xml in the resources folder to define Camuna engine settings. Finally, the most important file is the BPMN model file presenting our KYC process – kyc.bpmn. You can use Camunda modeler – a desktop application to create this BPMN model Drag and Drop interfaces.

3.4. Invoke a Java class from a BPMN task

Now we need to add the actual service tasks implementation using org.camunda.bpm.engine.delegate.JavaDelegate interface to trigger a REST API call to a corresponding service. Below is the StoreRequestService.java class to implement the “Store request” task.

@Component
public class StoreRequestService implements JavaDelegate {

  private static final Logger LOGGER = LoggerFactory.getLogger(StoreRequestService.class);
  private final RequestManagementClient requestManagementClient;

  @Autowired
  public StoreRequestService( RequestManagementClient requestManagementClient) {
    this.requestManagementClient = requestManagementClient;
  }

  @Override
  public void execute(DelegateExecution execution) {
    String userName = execution.getBusinessKey();
    requestManagementClient.storeRequest(userName);
  }

}

Since the input of our KYC process is “userName” so we will use the process’s business key as userName. Business key is a domain-specific identifier of a process instance. You can learn more about “business key” by Camunda official document https://camunda.com/blog/2018/10/business-key/.

In the above class, after getting “userName”, we will use it to make an REST API call using Spring Feign. In a similar way, we will create other JAVA classes to implement each task in the process.

VerificationRequestService.java
ApproveRequestService.java
RejectRequestService.java
ApproveRequestService.java
OnboardNewCustomerService.java

But how can the Camunda engine knows which task in the process a Java class matches? Don’t worry, Camunda allows us the specify the Java class implementation in the task’s setting. Using the Camunda modeler, we can easily do that by clicking on a task and then defining its implementation with a delegate expression is a Java class that implements the JavaDelegate interface.

Let’s dig deep into VerificationRequestService which is a special one because it needs to return and output to let Camunda know which is the next flow, reject or approve.

@Component
public class VerificationRequestService implements JavaDelegate {

  private static final Logger LOGGER = LoggerFactory.getLogger(VerificationRequestService.class);
  private final VerificationServiceClient verificationServiceClient;

  @Autowired
  public VerificationRequestService( VerificationServiceClient verificationServiceClient) {
    this.verificationServiceClient = verificationServiceClient;
  }

  @Override
  public void execute(DelegateExecution execution) {
    String userName = execution.getBusinessKey();
    try {
      verificationServiceClient.verify(userName);
      execution.setVariable("VERIFIED",true);
    }
    catch (FeignException exception) {
      execution.setVariable("VERIFIED",false);
    }
  }
}

We store to output into a variable “VERIFIED” and then use it to define the condition for each flow.

In the verify API I add an if-else logic to simulate how the condition works. So the process will go to the approval flow if the user name is “thoainguyen”, otherwise it will go to the reject flow.

@RestController
public class VerificationServiceEndpoint {

  private static final Logger LOGGER = LoggerFactory.getLogger(VerificationServiceEndpoint.class);

  @PostMapping("/verify/{userName}")
  public ResponseEntity verify(@PathVariable String userName) throws InterruptedException {
    if (userName.equals("thoainguyen")) {
      LOGGER.info("Verify success user name {}", userName);
      return ResponseEntity.ok().build();
    }
    else {
      LOGGER.info("Verify error user name {}", userName);
      return ResponseEntity.badRequest().build();
    }
  }
}

3.5. Demo

Now let’s test what we have created so far. We can start our demo application with gradle booRun, then access the Camunda tasklist web application via the URL with the default username/password demo/demo.

http://localhost:8080/camunda/app/tasklist/default/#/login

Then click Start process and select our KYC process, enter a business key which is the user name to start a new KYC process. Click Start.

You can observe the application log to know how the system works as below, you can see this process go to approve flow which means the user is verified successfully and is onboard.

2023-03-03 22:24:21.926  INFO 29432 --- [nio-8080-exec-8] c.t.r.RequestManagementServiceEndpoint   : Store request user name thoainguyen
2023-03-03 22:24:21.975  INFO 29432 --- [nio-8080-exec-3] c.t.rest.VerificationServiceEndpoint     : Verify success user name thoainguyen
2023-03-03 22:24:21.987  INFO 29432 --- [nio-8080-exec-7] c.t.rest.AccountServiceEndpoint          : Onboard account thoainguyen
2023-03-03 22:24:21.991  INFO 29432 --- [nio-8080-exec-1] c.t.r.RequestManagementServiceEndpoint   : Approve request user name thoainguyen

You can start another process with a different user name such as “tom”, then the log should be as below which means the user is not verified successfully so we reject the request.

2023-03-03 22:30:14.619  INFO 29432 --- [nio-8080-exec-1] c.t.r.RequestManagementServiceEndpoint   : Store request user name tom
2023-03-03 22:30:14.654  INFO 29432 --- [nio-8080-exec-9] c.t.rest.VerificationServiceEndpoint     : Verify error user name tom
2023-03-03 22:30:14.664  INFO 29432 --- [nio-8080-exec-5] c.t.r.RequestManagementServiceEndpoint   : Reject request user name tom

That is all for this topic, I just show you a simple example of using Camunda as an orchestrater in Microservices systems but it is the basic concept to start, you can refer Camunda’s official document to know its features as well as learn more about BMNP to apply Camunda to your project in right way.

The implementation of all of these examples and code snippets can be found over on Github

Using Camunda as an orchestrater in micro-services

1. What is Microservice Orchestration?

2. What is Camunda and how it works?

3. Microservices orchestration with Camunda

3.1. Tech stack

3.2. Simulate service’s APIs

3.3. Camunda integration

3.4. Invoke a Java class from a BPMN task

3.5. Demo

Leave a Reply Cancel reply

Kafka (Part 2) – Schema Registry

Kafka (Part 1) – Introduction

Resiliency Patterns in Micro-services