Pymma's Blogs
Featured

Data analytic: Apache Geode – A successful alternative to Kafka, Spark and Storm

Data analytic: Apache Geode – A successful alternative to Kafka, Spark and Storm

 

Pymma (www.pymma.com) is one the OpenESB project leaders (www.open-esb.net) and continuously works on OpenESB improvements and new features to offer the best Extended Service Bus on the market. Our Extended Service Bus covers a very large scope of applications from Internet of Thing, Integration of Services, Domain-Drive Design to Streaming Application or Event Processing.

One of the main OpenESB features is the orchestration of services based on a powerful BPEL engine. OpenESB BPEL engine generates a large amount of information about the running processes, useful for process analysis. Further to our customers' requests, we recently improved the BPEL engine to easily extract the information available during the process executions. Once our improvement completed, we evaluated that some of our customers could generate tens of billions of messages every day on their multi instances production systems.

OpenESB has not been designed to aggregate and analyse such amount of data. So, our architects decided not to rely on OpenESB to gather and analyse the messages coming from our orchestration system, but on an external application designed for this purpose.

This tiers application must fulfil four main requirements: ingest the flow of messages, put them together in only one place, aggregate the messages for further analysis then store the result of this aggregation on a persistence system queryable with a SQL like language.

 

...
Continue reading
356 Hits
Featured

Microservices VS SOA: The antagonism is not technical but..

Microservices vs. SOA war

Recently, browsing on YouTube, I re-watched the lecture given by Martin Fowler in 2014 on the microservices. As usual, Martin was a wonderful speaker, clear and concise. But in the middle of the lecture when he talked about differences between Microservices and SOA, I found his speech a bit more confusing and his conclusion on that topic was unclear and vague.

If you pay attention to his speech, before saying his conclusion on that chapter, he reported two witness statements: the first was from SOA people who developed successful SOA projects, surprised by the lack of innovation and freshness in the microservices definition and practices, and the second, from microservices people, mainly coming from big companies, fed up with SOA, who see SOA as 'Enterprise Services Bus' and 'committee that wants to lay down standards'. 

Then, Martin concludes by an evasive definition that Microservices is a subset of the SOA.

However, he did not give any border line between the two domains, but said that the term, SOA, is a too broad and the microservices is a useful subset of SOA.

With all my respect, I think that if Martin presented good arguments in his speech but did not give the real difference between microservices and SOA.

Before going forward, it is interesting to notice, that concept of microservices had a prompt success in large companies first. Surprisingly, from its inception, the microservices have been used to deprecate the 'SOA' concepts, even if the gap between both techniques was very thin or in many cases, null.

Here cHere ome a few simple questions: Why did some people try to oppose the two technologies that are so similar? Why did they want to push SOA to oblivion while they use exactly the same principles and technologies than SOA?

Martin provided in his lecture, many trails to reply to the questions, but I came to a different conclusion. If we can understand this antagonism which is not based on technical differences, maybe we will understand the differences between SOA and microservices.

To understand this antagonism, the SOA Manifesto, the abstract of the SOA philosophy, must be read once again. In the manifesto's introduction, it is written:

Through our work, we have come to prioritize:

Business value over technical strategy
Strategic goals over project-specific benefits
Intrinsic interoperability over custom integration
Shared services over specific-purpose implementations
Flexibility over optimization
Evolutionary refinement over pursuit of initial perfection

That is, while we value the items on the right, we value the items on the left more.

In this introduction, the first priorities were not technical challenges, but the Business value and the Strategic goals.

When you are concerned about Business value and Strategic goals in your project, it is in your interest to involve the most capable to define what the Business values and the Strategic goals are. This means that the SOA philosophy requires the business to collaborate and more annoyingly for IT teams, it needs a kind of leadership from the business, because Business value and the Strategic goals definitions are more accurate and efficient when defined by business people.

For years, business and IT lived in two different worlds. The communication between them was focused on budget, cost, business specifications and nothing else. It was OUT OF QUESTION that the business interfered in the IT works and the later in the business prerogatives. This antagonism between the two entities lasted for decades.

However, 10 years ago, for the first time in IT's history, a new philosophy puts the business goals and the long-term strategy as a priority. For the first time, IT people had the willingness to involve the business in the IT development process. If the business never understood what was a function, a component or an object, for the first time, SOA pushes forward a concept named 'Business Services', which had a coarse enough granularity to be understood and used by the business. So, for the first time, business and IT could exchange, and work on similar concepts.

Unfortunately, especially on larger projects, consulting companies found in this opportunity a way to impose expensive business consultants and management, to IT teams. I remember taking part in numerous meetings with high-rate business consultants who didn't have a concrete idea of what a service is, but could explain through wonderful PowerPoints, how SOA could help us to design and develop them. Pushed by the C level management and the consulting companies, a useless business predominance replaced a required and fruitful collaboration between both parties. A reaction, from the IT, to stop this predominance occurred and has been named 'Microservices'.

It is interesting to analyze, why this reaction took the name of 'Microservices' and not 'Light Services', 'Agile Services' or 'Reactive Services'?

The prefix 'micro' before 'services' that suggests services with fine granularity, does not make sense since neither SOA nor Microservices define a standard granularity for a service. Often services defined by microservices developer have a coarser granularity than the ones developed by SOA developers.So, what is the symbolism behind the prefix 'micro' and why it was so pleasant and attractive to IT people.

Wise men explained that a name, often contains the finality of the named entity or concept.

As noticed above, one of the SOA aims, was to promote Business Value and Strategic Goal. Consequently, this aim was to define services comprehensible by the business and coarse enough to be useful for business process definition. In the way IT thing, the services understandable and used by the business have a coarse granularity.

In reaction, IT people created the concept of 'Microservices'. The objective was to define small services where the granularity was too fine to be in the business scope. So, with microservices, the business cannot interfere in the IT work anymore. That is the opposite of the SOA initial philosophy.

Actually, there are no technical differences between SOA and microservices, but their fundamental difference is the interest in a relationship between business and IT.You can find this idea coming from a day to day psychology, but to be convinced, notice that neither the technical papers, nor the blogs, nor the conferences about microservices consider business concerns or simply talk about a link between microservices concepts and business concerns.

In reaction to the business predominance created by the bad use of SOA philosophy, business concerns and strategic goals have been erased from the microservices manifesto and developer objectives.

Microservices SOA Collaboration with the business Yes, please No, thank please
The border line between SOA and microservices is simple to understand and is defined by the IT wishes to work with the business. Through this dimension, Microservices is not a subset of SOA but a different domain.

Of course, the real life is not black and white and many microservices people collaborate with the business and some SOA people don't. Here, I'm not talking about people but the history, the philosophy and the concepts that generate the microservices wave. It is good to put names on things, and understand where SOA and Microservices are different. Up today, I did not find similar point of view expressed yet. I could be completely wrong, but I think, we are reluctant to this idea which associates microservices with the 80's-90's mentality and is not politically correct. So, it is difficult to express it before an IT audience who feel enthusiastic about microservices. Without this truth, we couldn't understand the increasing separation we notice today between IT and Business.

This said, everyone knows that the future of IT is to work in harmony with the business and allows it to be involved in service and process definition. We could wait for a few years, the next swing of the pendulum and then be thrilled by a new buzz name that promotes a collaboration between IT and business.

Or simply, whatever denomination we want to use for the same technology, we can retrieve a peaceful and successful collaboration that occurred a few years ago, when a strong respect existed between business and IT. That would certainly be the best choice for all of us!

790 Hits
Featured

Roger and his database

Introduction

Last week, I tripped in the Belgium and I met a nice guy called Roger (of course it is not a material public figure). In a bus, we had a friendly chat and after a while, he told me that he was an IT consultant and works as a DBA for a renowned insurance company.

For the ones who complain about their software editors, I would like to report the Kafkaesque story that Roger told me.

The context

Roger's company has used for years the ERP JD Edwards. Recently, JD Edwards has been bought by Oracle which has other ERP in its portfolio such as Oracle Applications. Roger manages and monitors Oracle DB V9 used by JD Williams. For months the EPR and the database worked perfectly together.

First act

Two months ago, for the first time, the database crashed. Roger is a good DBA and was able to quickly restore the database. Alas, afterwards a few hours the database went down again. Roger found out a sticky bug in the database and to ascertain a resolution, he shouted out the Oracle support which was not surprising since the bug was easily recognized. An Oracle consultant told Roger to change a parameter "P" from the value X to Y. "Perfect solution". The database worked well and did not crash anymore. End of the first act.

Second act

A few days after the cash episode, the users complained that the ERP performances decreased by 10 to 100. Quickly the company found out that the bottleneck came from the database. Another Oracle consultant stepped in to optimize the database and diagnosed the problem. THE SOLUTION was quickly found out: The parameter "P" set to the value Y must be set to X. Immediately, Roger reacted and replied that the value X crashed the database.

The solution

Oracle took the problem and suggested to migrate the database from the version 9 to the version 10. Indeed, the new version solved this problem.

Alas, the JD Williams does not plump for the version 10 of Oracle databases, even if the ERP and the database belong to the same editor. As a final solution, Oracle offered to simply drop, JD Williams from the company and rebuild the organization with Oracle Applications. Of course, Oracle would assist the company during its migration. (What a pitiful solution!!!).

Happy end

The CTO aware of this problem saw red and threaten the editor to withdraw all the Oracle products if a solution was not find asap. During the week, a patch was sent to Roger and all the issues were fixed.

I trust you enjoyed the depressing tale of Roger and his database...

2155 Hits