Event streams are the paradigm of digitalization of today

12. 08. 2020

Agility, the current shift in the use of backbone ESBs and their agile application, API microgateway, hybrid integration and the hot trend– event stream. Those are some of the up-to-date topics in the area of integration, a field that Trask has been focusing on since its very beginning. Petr Dlouhý, who has headed Trask integration competence for four years now, offers an insight into integration trends, benefits and pitfalls.

Trask presents integration and the latest integration trends on regular occasions, most recently at integration breakfast last autumn. Which short-term and long-term development do you see as the most important in the integration field?

The focus on agility remains one of the most significant shifts. Teams are often encouraged to implement application integration on their own (self-service integration). Such an approach entails numerous new challenges, for example the question of service ownership, mutual collaboration of teams, whether and how to approach reuse, and many others. I should say that there is no single, universal set of instructions how to handle such challenges. Each organization operates in a slightly different manner and the unavoidable cultural shift connected with the adoption of agile approach poses demands that are specific to the organization. Speaking about tools, such tasks require the implementation of an API management platform and the use of integration microservices, including containerization of run-time integration system (and potential move to cloud). Some of our clients already apply this approach in practice, others are in the stage of preparation or PoC, but the trend is visible everywhere.

Which trends would you point out for this and the nearest years and what benefits they bring or are likely to bring to clients?

Current interesting topics and trends are many. They are literally mushrooming, compared with the past, and it is always harder to follow and evaluate them, to apply them in practice and to make use of the gathered knowledge. Aside the general agile approach, ESB backbones and their use in agile environment are the focus of attention, as well as microgateway, hybrid integration and the hot news – event stream technology. The future development in the field of service registry and repository is another interesting topic, because many of the existing tools are not further developed or even not supported today. This is where analytics come in, and the monitoring of production environment actual status only, or perhaps tools such as HashiCorp Consul for better control over the microservice environment.

Event stream is another current topic that you have mentioned. Would you share your experience with us?

When SOA approach first emerged a couple of years ago, many organizations understood it as a panacea for any future issues connected with system integration and expected a good return of investment due to service reuse. However, the reality was different. Concrete approach and reasonable use were the key factor, irrespective of the numerous benefits regarding architecture, system decoupling and operational simplicity. Today, we see similar expectations with regard to event stream. It indeed is a very interesting approach/technology, but it does have its limits and one should consider very well and at the very start, for what purposes to use it and for what purposes not to. Also, do not forget that event stream entails a requirement for a shift in thinking and in integration design, including modifications of the very way integrated systems operate.

We have been paying attention to event streams for some time now, from log processing to real business applications. We have developed a reasonable approach to event stream and we now offer that approach to our customers.
Petr Dlouhý

You have mentioned that customers tend to divert from ESBs to integration microservices. Why?

I would not call that diverting. Rather, it is an alternative view of integration, based on the requirements agile methodologies place on the decentralization of integration and responsibility and on newly available technologies, such as microservices and containerization. The existing ESBs are transforming too, they can run as decentralized systems in containers, much like the alternative solutions that I have mentioned. It is interesting to follow, for example, the approach of Gartner to their backbone ESB. I remember that, four years ago, no one cared about an ESB, everyone was for API, microservices, hybrid integration platform. Suddenly they sobered up and, after two years, backbone ESB is back as one of their integration pillars, complemented by an API and integration microservices. Such an approach corresponds with our understanding of the matter.

If you look at a bank, one part of it is agile, fast changing, using numerous modern approaches (integration microservices, API, event stream). The other is stable, regulated, with ESB and messaging playing an indispensable role both now and in the future.
Petr Dlouhý
Petr Dlouhý

Petr Dlouhý

Petr has been working in IT sector for over 20 years, focusing on solution deliveries to financial institutions. He has joined Trask in 2012. He holds the position of Trask Department Director now, leading a team of 60 integration experts.

Event Stream

A comprehensive architectural and technological concept that deals with various issues related to request-driven integration, such as service availability, scalability, online batch, synchronization, notification, etc. In addition, it offers numerous novel approaches (data lake, data mesh, inside-out data, integration with analytics and machine-learning systems, etc.) supporting the existing design patterns (command and event sourcing, CQRS, materialized view, etc.). In Trask we see it as an important tool for integrations with complexity increase, functional and non-functional integration requirements. However, it does have certain weak points, such as event mining, the absence of ACID, data purity and consistence, data formats lifecycle, etc.

API Management

API management tools enable safe and managed publication of API interfaces and general web interfaces (REST, SOAP, etc.), including automated documentation in a Developer Portal. The market offers a whole range of ready-made API management products – clients can choose from commercial versions, open source, cloud and on-premise solutions. Most of them make creating and managing APIs easy, intuitive and self-service. API management is an indispensable feature for any organization wishing to expose its interface for external use. It is also extremely useful for internal API calls inside the organization. It is a pillar of integration in agile environment for many reasons, e.g. because it has a built-in support of authorization transfers and transfers of API publication responsibility to agile teams. Last but not least, it is an important addition to standard ESB-based integrations and it can easily deal with tasks that prove to be rather complex for the latter, such as authentication, authorisation, analytics, interface registers, or monitoring.

Interested in knowing more?