Microservice with Spring Boot and Docker with Weave (Full Example)

For the time being I'm working as a Docker Developer (not working on the Docker project).

I work for a company here in Gothemburg (Sweden) hired as a consultant.

This has given me a lot of opportunities to use Docker and learn its pits and falls.

Multihost:

One of the bigger problems have been multihost applications. To have containers on different hosts that talk to eachother.

One company that set out to solve this problem is WeaveWorks (http://weave.works/).
But the new experimental branch of Docker 1.7 has the new libnetwork so you can do multihost.

But as I said, it is in the experimental branch and some of it's functions isn't mature enough.
This is one of the reasons that I've been looking at Weave. Another is that the experimental branch of Docker also have plugin support. And Weave is one of the first to actually create a network plugin (many big companies are working on this).

So I can use Weave as I do now or I can use it later as a plugin.

Components:
Weave Binary that pulls Weave, WeaveExec, WeaveDNS containers (they run in containers).

So what are some pros with Weave:

• Mulicast (or Unicast) support
• The containers have there own IP so they are more like traditional containers
• Containers can have Hostnames that just works with WeaveDNS
• Password for your Weave network
• NaCl crypt
• Uses "Vanilla" Docker, so no real changes to your containers
• The network hasn't one point failure, start a network on one host, connect another to that host, connect a third to the second and they all find eachother and registers eachothers as peers.

Some cons:

• Bad documentation (when is crypt used and more advanced examples). The only documentation on how to use it is the simple guides, but no real guides for configuration.
• Not apparent what the Weave Binary does

I'm probably gonna have a session on the next Docker Meetup Gothemburg (Aug 2015) and maybe in

JavaForum Gothemburg.

So for these sessions I have prepared 2 demos. One with minimum code and one with more complete code (still very little code).

Demo

So the demo is 2 services:

Weave Producer:

The Producer is a Spring Boot Rest service that retrieves a Environment Variable and sends it. It listens on the url: host:8080/rest/hostname

This is packed as a standalone jar. I put it in a Docker container (extended Image from OpenJRE).

Weave Consumer:

The Consumer is a Spring Boot Rest service that retrieves a value from the producer and then adds some text. The result is meant to be which Consumer and which Producer is involved.

This is packed as a standalone jar. I put it in a Docker container (extended Image from OpenJRE).

Here comes the fun!

Since the Consumer doesn't know the exakt host it will talk to, it talks to a DNS-name.

weave-service.weave.local

weave-service is my name for the service and weave.local is the mandatory part. This part it takes from a properties file if I want to change it.

So a hostname might not seem like a magical thing, but the fun part is that you can start several containers of the same Image with the same hostname! 

WeaveDNS understands that it is the same service. I think (not sure) that it will loadbalance between them.

But what I do now is that if you have 1 Consumer and 2 Producers and the Producer that the Consumer is using for its service goes down (docker stop or unavailable) then (since I coded a retry function) it will automagically start using the other one (The DNS understands that it has gone down and will remove it from the DNS-list).

Weave'ing It

So the Demo is actually a Vagrantfile that starts 3 VM's running Ubuntu with Docker.

This takes up around 50-60 GB (sorry!).

The Github-page gives instructions on how to run them and build the Docker Images on the VM's.

All the code is included and I use Gradle Wrapper so you don't need to download Gradle or Maven for building the services. (You only need Java 8, the services are built on you host, not the VM's).

One VM will have a consumer and the other two will have producers. The Environment Variable that the Producer uses is set when you create the container (the name Producer 1 or Producer 2).

(Green is Weave connections, peers, and red is the DNS)

Conclusion:

A small and fun way to try out microservices with high availability using Docker and Weave.

Nothing in the Java-code does anything to do with the high availability part (just the retry but you want that even if you only use one host to talk to).

So for Weave to work you only need to use hostname, you don't need any other change.

Sorry for any misspelled things or typos.

Here is the Links:

Minimal Code: https://github.com/Khazrak/WeaveDemo

Better Code with fast Retry: https://github.com/Khazrak/WeaveDemoEX

Feedback:

I would love some feedback on these or some idés for improvments.

Happy Testing!

Docker: Hop on now or get passed by

So right now I'm actually working full time with docker as a consultant. The IT-division that I'm a part of has a plan to containerize their services and I'm the only one (in that division that's actually doing anything with Docker).

And I can say right now that NOW is the time to jump on to Docker and try it out, because soon many (not every one) will use it!

Scalability

I'm not pulling this prediction out of empty air. First of all, Docker makes scalability easier (not easy, but easier).
Docker-Machine can spin up a new Cloud-instance with Docker installed on many popular cloud vendors (Azure, Amazon, Digital Ocean, Google and more).

Docker-Compose can spin up a whole stack of Containers based on a config file (written in YAML).

Docker-Swarm can connect the Docker-machines in a cluster and soon it will seem as the whole cluster is one big host. They are getting there but Swarm is just 0.2-beta.

Windows is coming 

Docker-Client is now on Windows (http://azure.microsoft.com/blog/2015/04/16/docker-client-for-windows-is-now-available/)

This is the client so you still need a Linux Host to connect to, but you can actually control Docker from native Windows now.

Microsoft (with Docker.inc) is creating Hyper-V Containers and Windows Server Container to be side by side with Linux Containers through Docker.

They also have created a Docker Image (Linux-container) with ASP.NET 5 preview (https://registry.hub.docker.com/u/microsoft/aspnet/) and the move to "open source" parts of the .NET-platform means that some Windows-apps will be able to run in Docker Linux Containers.

Microsoft have also created a editor for cross-platform editing, Visual Studio Code https://code.visualstudio.com/

These pieces means you can code and run C# and Windows Apps from Linux/OSX and use them with Dockers scalability and isolation.

Conclusion:

And all this is happening right now!

So download Docker, try it, tinker with it and be ready for the ever developing world of IT.

More about Microservices

A long time since the last post.

I've now started a new job as a System Integrator, but I actually code Java the most.

So I thought a should ramble a bit about Microservices.

Pieces of a Microservice architecture:

Communication-technique

One of the big PROs with microservices is that they can be language independent. This however creates a demand for a language independent way to communicate.

Rest:
One of the most common ways right now is to make REST-services. Most languages have a httpclient-library to call services, usually urls built by Strings. For load-balancing you might need a services orchestration application, maybe Consul/Eureka/Zookeeper.

AMQP:
Another way is to use a message technique like AMQP. The AMQP-servers (Brokers) are very stable and have great clustering abilities. AMQP is a network protocol, which has it's perks. This way has less support but has almost automatic load-balancing (if several clients listen to one broker the use a round-robin strategy for messages). Request-respond actions can be a bit tricky (usually a queue for the request and then listen to a respond-queue).

Summary:
What do these have in common? They are stateless. This means you have to plan it in that way. Microservices demands a lot of modular thinking.

Data Persistance:

How about databases? How do we save data?
The important question is, how consistent must the data be?

One idea is to have a polyglot solution and have a datasource in every service which connects to eachother. Maybe MongoDB's sharding?

Another is to connect to a classic cluster of 2 or more databases.

Just be sure to not lock yourself in, don't forget that the individual microservices usually talk to the datasource, they need to be able to talk to them.

Another aspect is should the different services actually use any of the same data? If yes, then communication between teams (that code the services) needs to be top notch.

So preferably the answer is No.

Is DevOps a prequest for Microservices?

Microservice architecture has gotten a fresh start with the agile way of developing. And these small services are developed fast and change fast. There has to be a way to quickly deploy a new version of a service.

Many automation-tools have come together to solve the "pipeline", jenkins, go.cd, etc.

This is where Docker has made a good impact. If the service can be contained in a single container (or 2 if data is needed), it can move from development to test to production very easy.

But do we have time to teach a ops-guy how to administrate your service if it changes rapidly? If we need to scarp it and change the whole application-stack for another.

The developers are the experts, if they can administrate the service even in production, they have all the power to update, change and migrate the service.

So even if DevOps technically isn't a prequest for Microservices, it makes for a more agile way of developing.

Should I do Microservices?

The big question is SHOULD you do microservices?
Todays development is moving more to the modular architecture.
Which is a step towards microservices. But microservices are not a hail-mary solution. They come with a price, complexity. You need orchestration and communication-layer and a way to deploy all the services. They will probably take more performance too.

So what do we gain? Scalability and the option to quickly change parts of it.
Good modularity can also help against spaghetti code.

So my 2 cents if you should do microservices:

If you KNOW that you need to scale big some day or will aquire more functionality then microservices might be for you.

If your only doing a website, then microservices might only be unncecessary complex.


I will do a small Microservice project and put it on Github for a more concrete example, to show what I'm talking about. But it is not this day.

Happy Coding!