Abiodun Azeez
21 Nov 2022
•
9 min read
Microservices have recently emerged as the main architectural framework for managing cloud applications and solutions due to their ability to handle the scalability challenge.
This article will explain the notion of microservices and show you how to build one in several languages such as Go and PHP/Laravel that connect with one other over a message broker such as Kafka.
We are going to cover the following:
To follow along, you must have a basic understanding of the following:
A microservice architecture, often known as microservices, is a framework for decoupling a single application into a number of smaller apps or services, with each service responsible for its own domain action. The purpose of this design is for a service to be able to interface with other services simply using an API (Application Programming Interface) while also being able to be developed and deployed separately. Microservice architecture is most commonly used in complicated and massive systems like Netflix, Google, Apple, Uber, and others.
A monolithic architecture is one in which the frontend, backend, and database system are all developed as a single application with a single huge codebase. Most startups and organizations continue to adopt monolithic architecture because it is simple to set up and manage for small-scale applications.
Pros
Cons
Pros
Cons
Having a collection of services in a system necessitates a unified approach to communication. The major tradeoffs between these communication patterns are as follows.
Synchronous: This is a concept that interacts with other services via HTTP(Hypertext Transfer Protocol), RPC(Remote Procedure Call), and gRPC(Remote Procedure Call). This technique sends a request and waits for a response from the receiver.
Asynchronous: Another communications pattern in which a message is sent without waiting for a response; the message is processed asynchronously by the receiver. Because the sender does not have to wait for a response, this pattern is quick and efficient. Message brokers, AMQP (Advanced Messaging Queuing Protocol), and other protocols are presented as examples.
Monitoring of services is a vital component of implementing this architecture to ensure that everything works as planned and to prevent failures. A variety of things can be monitored, but the most important ones are:
Monitoring Tools (APM): Monitoring tools suitable for microservice architecture include Jaeger Tracing, Newrelic, Middleware, and others.
Setting up project
We will create a simple web page that will interface with other services for our project. The following is a breakdown of the services that will be developed and the technology that will be used to implement them.
Services Structure
Synchronous: REST API (HTTP)
Asynchronous: Kafka
Data Store: MongoDB
Deployment: Docker
All of our services, including Nginx, PHP, MongoDB, and Kafka, are included in the project's docker-compose file. In addition, a docker file is built for each microservice to micro-manage its build and dependencies.
version: '3'
services:
frontend-service:
build:
context: ./frontend-service
dockerfile: frontend-service.dockerfile
ports:
- "80:8080"
listener-service:
build:
context: ./listener-service
dockerfile: listener-service.dockerfile
restart: unless-stopped
environment:
kafkaURL: kafka:9092
topic: logger
groupID: logger-group
logger-service:
build:
context: ./logger-service
dockerfile: logger-service.dockerfile
ports:
- "3500:3500"
environment:
mongoURL: mongodb://mongo:27017
dbName: demo_app
collectionName: logger_db
broker-service:
build:
context: ./broker-service
dockerfile: broker-service.dockerfile
ports:
- "8083:1323"
environment:
kafkaURL: kafka:9092
topic: logger
php:
container_name: php
build:
context: ./auth-service
dockerfile: auth-service.dockerfile
env_file:
- auth-service/auth.env
volumes:
- ./auth-service:/var/www
ports:
- "9000:9000"
auth-service:
image: nginx:alpine
restart: unless-stopped
ports:
- "8082:80"
volumes:
- ./auth-service:/var/www
- ./auth-service/nginx/conf.d/default.conf:/etc/nginx/conf.d/default.conf
depends_on:
- php
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka
ports:
- "9092:29092"
environment:
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_CONNECT_BOOTSTRAP_SERVERS: localhost:9092
KAFKA_CONNECT_REST_PORT: 8082
KAFKA_CONNECT_REST_ADVERTISED_HOST_NAME: "localhost"
KAFKA_CONNECT_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
KAFKA_CONNECT_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
KAFKA_CONNECT_KEY_CONVERTER_SCHEMAS_ENABLE: 0
KAFKA_CONNECT_VALUE_CONVERTER_SCHEMAS_ENABLE: 0
KAFKA_CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
KAFKA_CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
KAFKA_CONNECT_INTERNAL_KEY_CONVERTER_SCHEMAS_ENABLE: 0
KAFKA_CONNECT_INTERNAL_VALUE_CONVERTER_SCHEMAS_ENABLE: 0
KAFKA_CREATE_TOPICS: "logger:1:0"
depends_on:
- zookeeper
mongo:
image: 'mongo:4.2.16-bionic'
ports:
- "27017:27017"
environment:
MONGO_INITDB_DATABASE: logs
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD: password
volumes:
- ./db-data1/mongo/:/data/db
Here is our Makefile (supported for Linux and Mac users), which is used to bootstrap our services and also aids in the creation of binary files for Go-based services.
LOGGER_APP=loggerApp
BROKER_APP=brokerApp
LISTENER_APP=listenerApp
## up: starts all containers in the background without forcing build
up:
@echo "Starting Docker images..."
docker-compose up -d
@echo "Docker images started!"
## up_build: stops docker-compose (if running), builds all projects and starts docker compose
up_build: build_logger build_broker build_listener
@echo "Stopping docker images (if running...)"
docker-compose down
@echo "Building (when required) and starting docker images..."
docker-compose up --build -d
@echo "Docker images built and started!"
## down: stop docker compose
down:
@echo "Stopping docker compose..."
docker-compose down
@echo "Done!"
## build_broker: builds the broker binary as a linux executable
build_logger:
@echo "Building logger binary..."
cd ./logger-service && env GOOS=linux CGO_ENABLED=0 go build -o ${LOGGER_APP} ./cmd/api
@echo "Done!"
build_broker:
@echo "Building broker binary..."
cd ./broker-service && env GOOS=linux CGO_ENABLED=0 go build -o ${BROKER_APP} ./cmd/api
@echo "Done!"
build_listener:
@echo "Building listener binary..."
cd ./listener-service && env GOOS=linux CGO_ENABLED=0 go build -o ${LISTENER_APP} ./cmd/api
@echo "Done!"
Frontend (UI): The frontend is a simple page that serves as an entrance point for users to engage with our application. It will be built with Vuejs and Bootstrap UI.
The Dockerfile for the Vue configuration is available here.
FROM node:lts-alpine
# install simple http server for serving static content
RUN yarn global add http-server
RUN mkdir app/
# make the 'app' folder the current working directory
COPY . ./app
WORKDIR /app
# install project dependencies
RUN yarn install
RUN yarn build
# RUN ls
EXPOSE 8080
CMD [ "http-server", "dist/" ]
Our user interface (UI) is simple, with three (3) buttons (test auth, test log, and get all logs) and columns for payload and output.
<template>
<div class="container">
<div class="py-3">
<h3 class="title">Microservice Demo <span class="fs-6 fw-normal">(Built with Vue, Go, Laravel, Kafka and MongoDB)</span></h3>
<div>
<div class="row align-items-start my-2">
<div class="col">
<h4 class="font-bold">Payload</h4>
<div class="py-4 px-4 payload">
<div>
<h6>Login</h6>
Valid details: <span class="badge rounded-pill text-bg-secondary">email: admin@gmail.com, password: verify</span> <span class="badge rounded-pill text-bg-secondary">email: user@gmail.com, password: user</span>
<div class="mb-3">
<input type="email" v-model="auth.email" class="form-control" placeholder="name@example.com">
</div>
<div class="mb-3">
<input type="password" v-model="auth.password" class="form-control" placeholder="password">
</div>
</div>
<div>
<div class="mb-3">
<label for="log" class="form-label">Enter Log sample</label>
<textarea class="form-control" id="log" v-model="log" rows="3">This is a log sample from frontend</textarea>
</div>
</div>
</div>
</div>
<div class="col">
<h4 class="font-bold">Output</h4>
<div class="w-full border overflow-auto break-words output-h">
<pre class="">
{{response}}
</pre>
</div>
</div>
</div>
<div class="mt-10">
<button class=" p-3 btn btn-success rounded-md text-white" @click="handleAuth">Test Auth</button>
<button class="mx-1 p-3 btn btn-primary rounded-md text-white" @click="handleLog">Test Logger</button>
<button class="p-3 btn btn-dark rounded-md text-white" @click="handleGetLogs">Get Logs</button>
</div>
</div>
</div>
</div>
</template>
<style scoped>
.title {
color:# 666;
border-bottom: 3px solid# DDD;
}
.payload {
background:# EEE;
height:500px;
}
.output-h {
height:500px;
}
</style>
Each button has its own event, which calls the broker service with its own payload. According to the docker-compose settings, our broker service is accessible at http://localhost:8083.
<script>
import axios from 'axios'
import { ref, reactive } from 'vue'
export default{
setup() {
const response = ref()
const log = ref('')
const auth = reactive({
email: '',
password: '',
})
const handleAuth = async () => {
if(auth.email == '' || auth.password == '') {
return alert("kindly enter email and password ")
}
log.value = ''
const payload = {
action: "auth",
auth: {
email: auth.email,
password: auth.password
}
}
try {
const res = await axios.post('http://localhost:8083', payload)
response.value = res.data
auth.email = ''
auth.password = ''
}catch(error) {
response.value = error
}
}
const handleLog = async () => {
if(log.value == '') {
return alert('Kindly enter a log')
}
auth.email = ''
auth.password = ''
const payload = {
action: "log",
log: {
name: "log",
data: log.value,
}
}
try {
const res = await axios.post('http://localhost:8083', payload)
response.value = res.data
log.value = ''
} catch(error) {
response.value = error
}
}
const handleGetLogs = async () => {
const payload = {
action: "logs",
logs: {
}
}
try {
const res = await axios.post('http://localhost:8083', payload)
response.value = res.data
}catch(error) {
response.value = error
}
}
return {
log,
auth,
response,
handleAuth,
handleLog,
handleGetLogs,
}
},
}
</script>
Broker Service: This is an API gateway that stands between user requests on the frontend and our services, acting as a single entry point for all requests from the frontend and responses from our services.
It will be written in Golang.
FROM alpine:latest
RUN mkdir /app
COPY brokerApp /app
CMD ["/app/brokerApp"]
Logger Service: The Logger Service logs data to MongoDB. It will be written in Golang.
FROM alpine:latest
RUN mkdir /app
COPY loggerApp /app
CMD ["/app/loggerApp"]
Auth Service: This is a service that will be created in PHP/Laravel to handle simple user authentication. To begin, we must consider a web server, which in this case is Nginx, PHP, and a PHP extension for Kafka.
FROM php:8.1.1-fpm-alpine
RUN apk add shadow && usermod -u 1000 www-data && groupmod -g 1000 www-data
RUN set -eux; apk add libzip-dev; docker-php-ext-install zip
RUN apk add --no-cache --update --virtual buildDeps autoconf
RUN docker-php-ext-install pcntl
RUN docker-php-ext-configure pcntl --enable-pcntl
ADD https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions /usr/local/bin/
RUN chmod +x /usr/local/bin/install-php-extensions && \
install-php-extensions rdkafka
RUN apk update \
&& php -r "copy('https://getcomposer.org/installer', '/tmp/composer-setup.php');" \
&& php /tmp/composer-setup.php --no-ansi --install-dir=/usr/local/bin --filename=composer \
&& rm -rf /tmp/composer-setup.php
WORKDIR /var/www
COPY . .
EXPOSE 9000
CMD ["php-fpm"]
For our authentication service, we establish an API route to handle our login, as well as a health check endpoint in case we need to verify the service status.
When a user signs in, the AuthController performs the login action in an invokable method that also publishes a topic(logger) or an event to a listener, and the listener then calls the logger service through REST API.
Route::post('/', AuthController::class);
Route::get('/health-check', function() {
return response()->json(['status' => 'OK']);
});
Listener Service: This is a service that listens to and processes the queue events triggered by other services. It will be implemented in Golang
FROM alpine:latest
RUN mkdir /app
COPY listenerApp /app
CMD ["/app/listenerApp"]
The final outcome of our project may be found here: https://github.com/iamhabbeboy/microservice-app. Make sure docker is installed before running make up_build
.
In conclusion, there's a reason why microservice architecture is so popular and used by large corporations. I'd say it comes with a lot of benefits at a high cost, and if not managed properly, it could cause a disaster, as it's difficult to figure out when working with about 800 or more services calling each other all the time, as Netflix does. At this stage, with so much data traveling around, the monitoring visualization tool may become overwhelming.
As a result, before choosing microservice design, make sure you exhaust all other possibilities.
Thanks for reading ✌️
Ground Floor, Verse Building, 18 Brunswick Place, London, N1 6DZ
108 E 16th Street, New York, NY 10003
Join over 111,000 others and get access to exclusive content, job opportunities and more!