Enterprise Integration – Back to the Roots

Just recently, I found myself getting involved again in more and more conversations around Enterprise Integration. Integration is nothing ‘old’ or ‘outdated’ I learned. It’s crucial still to optimise your business processes, reduce your cost base, be agile and run a profitable business.

As a matter of fact, newer paradigms like Big Data, Internet of Things, Predictive Analytics, Data/Information Services, Data Virtualisation, Data Firewalls, Hadoop or Microservices actually rely on a good integration architecture to provide business value.

Many  conversations seem to be driven religiously around technical / syntactical (meaning SOAP vs ReST vs Messaging) integration. I think this is only the 2nd best way to approach integration.

Keeping business scenarios in mind often brings clarity to what the best way is to ‘do’ integration.  Thinking holistically from Business Event generation upstream, the business process involved and all the way down to your Big Data lake to provide reporting and analytics is key to make the right architectural decisions.

Here are short videos on Service Orientated Architecture & Enterprise Integration in general by John Schlesinger:

Courtesy of J. Schlesinger – who I had the pleasure to meet while working in the banking industry on Globally Scalable Enterprise Integration Architectures  (and who I regard as my guru in the realm of Enterprise Integration) I am publishing his great papers here:

Enjoy!
Andreas

Business Oriented Enterprise Integration

After having examined low-level concepts and implementation specifics of Scalable Enterprise Integration Architectures earlier, this article is exploring the higher-level concepts and how it all (EDA, SOA, REST, Message Queues, Request/Response that is) comes together.

We shall call it ‘Business Orientated Enterprise Integration’ (BOEI).

Baffled or Just NOT Aligned at all?

Those of us who love to play ‘Buzzword Bingo,’ during meetings or conferences, know that the phrase ‘Business and IT Alignment’ scores frequently. I am surprised how many organisations still disregard it after having spoken about it for so long (Maybe we are secretly chasing high scores at Buzzword Bingo still? 😉 )

How do you know whether IT is aligned with business?
Does the organisation have an Enterprise Architecture (EA) practice – or maybe a vision statement from the CEO, with buy-in from senior & middle-management? Or a strategic roadmap that everyone knows about? What about reference architectures, published standards?

If the answer is No to most of those questions above, I doubt you’ve got business and IT alignment, simply because you have no way to execute according to common sets of standards and rules.

TOGAF talks about things like Architecture Repository, Enterprise Continuum and references to industry or organisational standards, patterns, guidelines and rules. Establishing those allow organisations to walk in the same direction as one entity, by reviewing initiatives according to those standards. And Yes, I am optimist enough to assume that people from the business end of town are communicating with the guys with the thick glasses in the dark rooms to discuss those matters together.

Divide & Conquer: Defining your Business Domain

Business-Oriented Enterprise Integration works for organisations of all sizes. Once you reach a certain size or your business is organised naturally in distinct units you might want to consider dividing the whole into more manageable chunks, or Business Domains, as depicted in the picture.

The same is true for M&A scenarios where you either want to sell parts of your business or integrate acquired businesses more easily.

Example of a Domain

A smaller Super Fund or Retailer might just have a single domain, whereas a globally operating bank can have separate domains split up by country, payments, institutional or retail banking.
Even a ‘Customer’ domain providing a single-view of customer to your organisation or a single-view of your organisation to the customer, is possible.

The major point however is: The Business defines a domain, its capabilities, the systems contained within and hence its boundaries. The business also owns the Domain and the systems, not IT (collaboration models between IT and Business, e.g. running IT as a charged service centrally, project based or as part of a business unit is another topic).

The Core: Semantic Hub

One of the major advantages of the BOEI approach is semantics.
Service Orientation forces core business systems to understand the semantics of the world of the target system or at least the ESB, which is NOT it’s job.

For example:
Core System A only deals with Account ID and Last Name; it then needs to understand how that translates into the ESB (canonical) or worse (because it is point to point) target system world. If canonical services are employed, each Core System Service has a Canonical representation of the very same service on the ESB, which doubles the amount of services (see ‘EDA vs SOA’ for details) and Version Hell has arrived.

The Semantic Hub decouples Source and Target Systems through well-defined message formats and the 3-forms-2-transforms pattern (see John Schlesinger’s ‘The Five Golden Rules of Integration’ for details).

The 3 message formats (forms) are:

  • Source System Message
  • Canonical Message
  • Target System Message

The 2 message transformations occur between the

  • Source System Message format to the Canonical Message format
  • Canonical Message format to the Target System Message format

It sounds very simple, but let’s be honest: Message modelling is an art that needs to be understood, especially when complex 1-to-Many, Many-to-Many or reverse entity-relationships are modelled (i.e. Customer-to-CustomerAddress or Account-to-Customer and Customer-to-Account).
[Please contact me and share your experience with tooling in this area!]

Message formats are contracts between systems owners (business people that is). Those formats need to be adhered to. The Business Domain owner facilitates the definition of those formats and enforces compliance through the Semantic Hub. The compliance comes naturally because it is the foundation of an operating business.

System Design and System Integration

For all the reasons discussed in ‘Event Driven VS Service Orientated’ the main integration and communication pattern between systems is Event based.

Because Businesses are driven by events. Naturally. Purchase order, invoice, delivery, payments, etc is what makes businesses tick.

BOEI thinking affects system design, because in an Event Driven Approach (EDA) systems are expected to do their job commonly without depending on anyone else.
That means a system’s data model must be able to store all the data necessary. For example if the Account System needs customer data to verify against when accounts are altered, closed or opened then that system must have that data available. A getCustomerData service call to a CRM system would defy the scalability advantage as discussed in ‘EDA VS SOA’.

The downside of this approach is that data might be stored redundantly. This is counter acted by the fact that the business owns the systems and it’s data and hence decides on what data to store plus the level of data quality necessary for the system to function accordingly.

The Fringe: Inter-Domain Communication and Presentational Systems

Inter-Domain communication adheres to the same principles as system-to-system message exchange within a domain (intra-domain).

This means event messages from source systems must be routed through the Semantic Hub before the message reaches an inter-domain gateway. Otherwise it would be a point-to-point integration between system and a domain. And the principle we must adhere to is loose-coupling.

Information retrieval for Presentational or Channel systems is commonly powered through Request-Response type communication. Above picture shows how we take the load off core systems by feeding presentational applications through Information Services. The data repositories that power Information Services are fed by canonical event messages directly from the Semantic Hub. Those data stores can be optimised for read-operations.

REST and JSON further enhance information retrieval through a standardised way to query resources (entities) and reduced response payload.

Functionality (e.g. Internet Banking money transfer) beyond simple information retrieval is implemented by re-using existing message formats from inter-systems integrations as described above.

How is this (more) Business Oriented?

  • Systems and domains are organised according to business operations
  • Business events are used naturally and not ‘clouded’ trough artificial services.
  • Systems and domains do not have to understand the semantics of other interfacing parts of the business with different semantics
  • The business owns systems and domains
  • Business domain owners enforce message compliance naturally
  • Systems do not depend on other systems to do their job
  • Core business functionality is decoupled from Channels

Is BOEI different to what you have got at the moment? Then ask yourself: Where is the money going to come from (Vision & Business Architecture Phase within TOGAF) ?

Look at what your current architecture offers and how it restricts or empowers the business (Baseline Architecture). Then define your Target Architectures according to the Vision & Business Architecture. Finally put priorities on the tasks coming out of your Gap Analysis.

That should give you a good understanding of what’s next on your agenda.

Conclusion

The approach described in this article is not new. It’s simply Best-Practices applied across an enterprise, using proven methodologies.
Unfortunately, in quite a number of organisations it seems as if the incumbent software vendors have a strong say on Strategy & Architecture, pushing for the latest software stack. And because most vendors have been very busy in recent years to make their entire software stack SOA ready, they don’t really want to talk about anything other than SOA.

But there was a reason why Message-Oriented-Middleware and Message Queues were invented in the nineties. And they still are a great team! And that’s why we should favour proven and purpose built technology to equip our organisations in the best possible ways.

The BOEI concept hopefully encourages you to have a closer look at your organisation, it’s goals, architectures, systems, data and processes in order to help you choose what will work best for you – instead of blindly following pretty marketing slides.

Selamat Tinggal from GA715 (just about to leave Northern Territory now),
Andy


Groundhog Day with SOA Architects

Sometimes I feel a bit like Bill Murray in Groundhog Day (except he gets paid better) since I started to propose, what I believe to be a better alternative, to pure Service Oriented Architecture as an approach to Enterprise Integration.

Let’s say the meeting starts at 10am and I arrive on time (because I am German and therefore I can ;-)) and then it goes like this:

Other: Hey Andy, I heard you are against SOA, what’s wrong with it?

Me: There’s nothing wrong with SOA there’s just a better alternative for Enterprise Integration.

Other: Better in what?

Me: Scalability for example, SOA doesn’t scale as well as it should if it’s the backbone of an Enterprise Integration strategy.

Other: What do you mean, we just need to add some more nodes to our ESB cluster, if load increases.

Me (Thinking: That’s exactly what I am talking about!): It doesn’t scale well, because it mainly relies on Request/Response which creates double the amount of network latency, it depends on the Service providing system to be up and running and it uses distributed transactions or compensation flows and hence requires undo-service implementations. And you can’t predict the amount of service calls.

Other: That’s why SOA recommends a strong Governance, so that we know who the consumers are.

Me: You are still not able to predict the exact number of calls at any given point in time, plus once your service endpoint is made public there is little enforcement for who calls it.

Other: But you can force authentication/authorisation on service calls.

Me: I don’t think authentication & authorisation should be part of a standard Integration scenario inside an Enterprise.

Others: Services don’t have to be Request/Response, a [one-way] queue can be a service endpoint too!

Me (Thinking: I wonder how a getCustomerData service call might get the customer data back then!?): How does a getCustomerData service return the customer data then?

Other: <Silence>. But it doesn’t need the service providing system to be up and running if you put the service request in a queue.

Me: But then you will have to correlate the asynchronous response to the request each time. The underlying issue with SOA is that it depends on other systems to do something, whereas the Event Driven approach doesn’t.

Other: We can just use an Orchestration Engine to support long running tasks.

Me (Thinking: Funny that those Orchestration Engines always show up in a integration discussion!?): Orchestration introduces state, and that’s no good in Enterprise Integration, because it doesn’t scale and failover easily and hence creates unnecessary overhead.

Other: You don’t want a BPEL Engine!? How do you orchestrate business processes then!?!!!

Me (Thinking: Oh no, not again!): Event Driven Architecture doesn’t need an Orchestration Engine. Whenever a business event happens, all systems that are interested in that event subscribe to it.

Other: But what about this process: <Other describes passionately a business process>…

Me (Walking to the whiteboard, throwing the first 2 markers in the bin because they don’t work): <Whiteboarding>…

Other: But what about that process: <Other or friend of other describes a more complex business process more passionately>…

Me: <Whiteboarding continues>…

Other: Ok, that was an easy example, but what about that one …

Me: <Whiteboarding continues>…

(The above “But what about/whiteboarding”-loop happens usually between 2 and 5 times) until “Other” or “Friends of Other” cannot come up with a more difficult process.

Other: I will have a think about it and come back to you with an example that cannot be solved without a BPEL Engine.

Me (Thinking: No, you won’t): Sure.

We are now already somewhere between 30 and 90 minutes into our meeting time, we have been kicked out of a meeting room at least once, because other people have booked the meeting room after us, but now…

…the meeting finally starts!

Have a great day!

Successful SOA Projects

Frankly speaking, I am a bit confused.

Gartner is saying that 75% of SOA projects are failing, whereas others (especially those who work on SOA projects) are claiming that they have delivered successful SOA projects.

What are we trying to do?

From previous articles you might remember that I am a believer in context: Are we using SOA for Application Integration? Are we writing new applications? Or is our goal to Integrate an Enterprise?

However, in this case the different contexts have a lot in common if it comes to quantifying and qualifying the success of SOA.

The following questions should help you to examine whether a SOA rollout is successful or not:

  • How many Services are there in total (provider and canonical form)?
  • Are there enough services to actually call it a SOA based solution?
  • What was the total build time per service?
    Maintaining the current speed, is it a good enough TTM?
  • How many of those services are re-used across Enterprise/Domain?
    If the solutions are just departmental, why are you calling it ESB then and where is the benefit of re-usability gone?
  • Do Services exist in an original and canonical form?
    If there is no canonical representation of a service you are tightly coupling domains and their systems.
  • How is the Service lifecycle governed?
    IBM has found that SOA Governance is necessary for a successful project; you might want to stop and (re-)define your governance first.
  • Who owns those services and how is a change governed?
    Who is paying for the services? And what happens if composite services have different SLAs in place? You will pay for the SLA upgrade?
  • Who owns orchestrated services which use services from different owners?
    (see above)
  • How many versions per service are there?
    Has version hell already begun to manifest itself?
  • How is transactionality handled across services (WS-AT, No Tx)?
    Do scalability tests and find out whether the current approach is scalable enought – the sooner the better!
  • How is compensation handled? How complex is it?
    Often compensation handling is more complex than the services itself. If that is the case you need to stop and ask whether the current SOA approach will be working for you in the future.
  • What do those services do? Execute functionality (Write) or query information (Read-only)?
    Information/Query services are ok for representational gateways, not if they increase system inter-dependencies. If service calls change state, check out the point on scalability and transactions (above).
  • How is determined what the max usage load will be during peak?
    You will need that to determine whether your approach is future proof.
  • How many service consumers are there per service?
    If there are only 1 or 2 consumers per service, ask yourself  Why? Are there too many versions, is it not re-usable or not mature enough yet?

If we get answers to above questions, then we have a pretty good understanding where the SOA rollout is at:

  • Are there only 3 read-only services (too early to make a call)
  • Do we have 10 different versions per service (version Hell is already on it’s way)
  • There are no clear ownership boundaries and contracts in place (Governance issue)
  • Etc.

Hope all is well!
Andy

Scalable Enterprise Integration: Events or Services – A Comparison

I am currently finding myself having discussions about the best way to provide globally operating Enterprises with a scalable Integration Architecture.

There is the Event Driven Architecture (EDA) approach, which has been around and proven for quite some time now. I believe it was one of the reasons Message Oriented Middleware was born and why IBM came up with MQ Series in the early 90s.

On the other hand there is the Service Oriented Architecture (SOA) paradigm that is nowadays lived and breathed by many vendors which have ‘SOA Ready’ software stacks to sell.

In order to not get opinionated for the wrong reasons, I had a look at the basic principles and underlying technologies.

And I was quite surprised by the result…

Consistency Model: BASE VS ACID

Verdict: EDA.
According to the CAP theorem, out of network partitioning, availability and consistency you can only achieve two at the same time. That is why Amazons IaaS offering applies BASE (Basically Available, Soft state, Eventually consistent) across their globally distributed infrastructure.
A global enterprise faces similar challenges. Where systems are distributed across countries and continents, BASE is the way to go in order to integrate a global enterprise.

ACID (Atomicity, Consistency, Isolation, Durability) is important for applications and systems which have immediate consistency as a paramount objective.

Scalability: High VS Low

Verdict: EDA.
SOA is emphasising on service calls, which creates dependencies on all service provisioning systems to be up and running and requires distributed transactions (e.g. WS-AT) or additional compensation flows. It is hard to predict how many service calls a service provider needs to cater for during peak load.

EDA uses local transactions to put event messages on a queue. The routing of messages is done via managed subscriptions, which allows to examine the exact number of subscribers.

Communication: Asynchronous & Message Oriented VS Synchronous Request-Response

Verdict: EDA.
Albeit it is true, that SOA can be operated over messaging infrastructure (e.g. SOAP/MQ or SOAP/JMS) the real problem lies in the request to do something and the associated wait for a response, which means systems depend on other systems to complete a task and respond (see also scalability).

Another drawback on SOA is that it is impacted twice by network latency, because there is always a service request plus it’s response whereas EDA doesn’t need a response if there is none required.

Messaging Infrastructure Setup: Complex VS Simple

Verdict: SOA.
On a pure Messaging Infrastructure level SOA/HTTP wins over EDA/MOM, because of technology like DNS, which help to automatically resolve service end points. To be completely fair though we need to note that we are really comparing two  pre-existing infrastructure assets (HTTP/DNS) with Message Queues, which is more a foundational layer of both paradigms. However if you go down either of those two routes you need to be aware of what you can re-use.

Endpoint Layer: Infrastructure VS Application

Verdict: EDA.
In terms of (OSI type) layering, a Service endpoint requires more layers then a message endpoint.

A Service usually requires network, DNS, platform (Java/.NET), App Server, Web Server.

A message endpoint requires network and a queue infrastructure.

Ownership: System Owner VS Multiple

Verdict: EDA.
Because SOA believes in re-use through orchestration, an orchestrated service can involve multiple services provided by multiple system owners.
This can create conflict in terms of ownership and responsibility to adhere to a specific Service Level Agreement (e.g. Platinum Services).

In EDA the system owner of an event message emitting system is clearly defined as the owner of that message. Any type of changes to that message need to be agreed on with this owner.

State: Stateless VS Stateful & Stateless

Verdict: EDA.
Let’s be clear: In Enterprise Integration, you do not want state. Application architecture might need state, but even there, a good application architect will try to avoid it because it is expensive and does not scale or failover easily. That is one of the major problems with service orchestration as soon as it requires compensation flows or distributed transactions. And that is why a Service Orchestration Engine might be of great use as part of an application or system, but not in Enterprise Integration.

In Case of Error: Local Transaction Rollback VS Distributed Transaction Rollback or Compensation Flows

Verdict: EDA
EDA relies mainly on local transactions for message delivery and pickup. Even if there are transactional clients involved the local system context remains, because – unlike SOA – there are no calls to distributed services and systems necessary.

SOA compensation flows add complexity through ‘Undo’-services in original and canonical form.

Decoupling: Canonical Message Form & Hub VS Canonical Service Endpoint & Service Bus

Verdict: Draw

Interface Contract: Message VS Service

Verdict: Draw

Provider Uptime Dependency: No VS Yes

Verdict: EDA
Another major drawback of SOA as an Enterprise Integration approach is it’s dependency on all the service provisioning systems. The service providers need to be up and running and must have enough spare capacity to service the request.

EDA on the other hand notifies other interested systems of an event that has happened. Those systems are free to pick up the event message when they are ready and more importantly it does not prevent any other system from completing it’s task.

Consumer Uptime Dependency: No VS Yes

Verdict: Draw/EDA.

If the service response is delivered asynchronously then SOA and EDA behave in the same way.

In case you are using synchronous service invocation (e.g. over HTTP) the EDA response message can sit in a queue until the receiving system is ready to pick it up, whereas the SOA response might never be received.

Transaction Handling: Local VS Distributed

Verdict: EDA.
See “In Case of Error” above.

Message/Service Peak Load Management: Subscription Based/Predictable VS Random/Unpredictable

Verdict: EDA.
If a system X does something (e.g. Account create) 500 times per minute then we know that we will receive 500 notifications during that minute. The Integration Hub knows how many receivers this event notification has because systems subscribe to it.

If system X now calls a service 500 times then the provider of this very service might or might not have the capacity to deal with it because there is no reliable way of predicting how many service calls that provider will get during the minute from other systems.

Network Traffic & Latency: One Way Message Delivery VS Request/Response

Verdict: EDA.
Event Driven Architecture, in most cases, delivers a message to the underlying messaging infrastructure and is done (fire & forget).

Service Oriented Architecture  works in a Request/Response paradigm and is hence affected twice by network latency.

De-Comissioning of Event/Service Provider: Simple VS Potentially Complex

Verdict: EDA.
Because there is no clear understanding of who is calling particular services, turning off services might have unwanted effects across the enterprise.

In case an Event emitting system has been turned off the event simply will no longer be received by other systems. Alternatively the same event can be emitted from another system if the business deems that event necessary.

Summary

It’s a quite clear Pro-Event Driven Architecture result, which is a bit of a surprise given the fact that there is still a lot of hype and products (Which one was first – hype or product?) around SOA.

But it does not mean that SOA is inherently wrong or faulty. The context for this comparison is very clearly set to ‘Highly Scalable Enterprise Architectures’ for globally operating Enterprises. There might still be a valid reason to go SOA if the context provides a fit.

Cheers,
Andreas Spanner

Related Links:

Banking, Integration Architectures and Customer Data Ownership

Gartner Analyst Alistair Newton raised data ownership as one of the new main concerns for CIOs in the Banking industry. His view ecompasses handing value-added data back to the Customer for re-use on other related services. Banks need to be managing data internally in a way that the customer can access and update the part that he/she knows best about, like Name and Address for example.
This imposes demands on a banks ‘Single View of Customer’. More importantly it demands (the right!) MDM, Data Quality and Integration Architecture.
In this article we will explore different types of Integration Architectures and come up with a best fit for purpose.

A New Paradigm

“Customer Own Their Data” is quite remarkable considering the MDM and Data Quality discussions in recent years. Not everyone was convinced a MDM project is necessary at all. A lot of focus was put on Functional Integration (i.e. Services/SOA), less so on true Data Integration (MDM/DQ).

For us Customers to own our data makes sense. If my name is ‘Andreas’ but I want to be called ‘Andi’ – from a Customers perspective the bank should comply with that wish. If you then look at the systems landscape inside of large enterprises only few seem to be ready to satisfy a simple wish like this. And the reasons for this are plentyful:

  • Failing or incomplete SOA projects
  • New data aggregation applications (e.g. web portals) are built, instead of integrating the data backends
  • IT projects overrun in time and budget
  • Business and IT run as disconnected entities

So?

Firstly, all the systems which require Customer data need to use the same data records of the entities, which are owned and maintained by the customer.

Why? Because, if I change my name now from ‘Andreas’ to ‘Andi’ through the Online banking interface, I’d like to see that change on my next account statement, too.

In order to do so, we need either

  1. Shared Data or
  2. Data Update Notifications

It is important to note that a central MDM Master is now becoming the MDM slave for the Customer owned entities (like Name and Address for example). Those MDM entities are now ‘mastered’ by the Internet Banking system.

‘Shared Data’ means there shouldn’t any Customer data, except a global identifier, be lying around anywhere else. An MDM master can be such a data repository, but making all the legacy systems using the MDM master is a whole different story.

‘Data Update Notification’ means I am receiving a notification that Customer data has been updated.
Best Practice is to send the whole updated record around instead of just the identifier. This is to save the unnecessary overhead of subsequent getCustomerData requests from all insterested systems.

This is what it means for systems to use the same Customer data. And it is where an Enterprise Architect should start to think about scalability.

Scalability

In a purist SOA/ESB way we would create a single service to set&get the latest and greatest customer data. This service would ideally be unique across the whole enterprise. The peek&idle load of those service calls from all enterprise wide service consumers would be hard to predict and you could end up with the data repository access as the bottle neck. Sure caching is an option, but then you need to think about cache location, cache-size, examine cache hit and miss counts and cache-invalidation strategies. And I’d argue then that this is not part of the core problems you should spend a large portion of your thinking on, when solving your Enterprise Integration challenges.

Remember, aim is to make the Account Statement System print ‘Andi’ instead of ‘Andreas’. If the system, which the Account Statement System uses to get the latest Customer data, is down during my update, the service call and hence my name update will never make it onto my statement. Best case senario is that I will get an error message like “Dear Valued Customer, Due to an Internal Error the requested operation cannot be completed. Please try again later”. It should perhaps read “Due to the wrong Enterprise Integration Architecture the request cannot be completed.”

Solution Architecture

How about sending a ‘Customer Details Updated’ notification to all systems that hold/store my customer data? There will be only one notification per Customer update, no unmanageable sub-sequent service call load, caching has become unnecessary and if a system is down the message will be delivered through a queue whenever the system comes back online

The thinking behind these two Solution Architecture approaches  are quite different. The SOA approach is based on the ACID paradigm whereas the EDA (Event Driven Architecture) approach is based on BASE. In case you want a refresher, Werner Vogel (Amazon CTO) has written a great article on that topic.
The ACID approach is perfectly valid in a non-distributed Application architecture, on an Enterprise Integration Architecture level things change.

Ready to hand the data- ownership over to the customer?

It seems as if the new way to go is to hand data-ownership over to customers. Customers should be happy and CIOs have to worry less about the Match&Merge step withing their MDM and Data Quality projects, at least for the Customer owned entities. However, as shown above, it is still paramount for CIOs to get the Enterprise Integration right first – and that is nothing new.

What’s your view?

Andreas Spanner

Enterprise Service Bus (ESB) – A Great…Marketing Term

When you talk to CIOs, one of their major concerns is Integration. CIOs wish everything should just magically work together.
Middle to large scale enterprises are more often than not organised in silos. Years of operation has made them so in order to get things done and to stay manageable, operational and hence in business.

To ease the pain of those poor CIOs the vendor land gang has hired some Genius Marketing Gurus to come up with a really appealing term, something that literally sells itself, and so they did: Enterprise Service Bus.

What a great concept! A single bus connecting the whole enterprise! The CIOs integration problems all solved! Where do I need to sign?
Oh, hold on, did the marketing presentation mention that the binary message formats of existing messaging solutions like TibcoRV, IBM MQ, MSMQ, SolaceSystems are not compatible?
No? Ok, the message formats are not compatible. But why would a CIO care, isn’t that binary message format waaaaay to low-level to be important.

I don’t know. What do you think?

I think it is important, because it means that there is really nothing like an Enterprise Service Bus, at least not from a product perspective, unless you do a rip & replace.
But then the CIO will need to decide which ones to rip and which one to implement and then he or she will be locked in to a single vendor – for life. Which is not pretty either.

What about the architectural concepts of an ESB then?
Let’s say we simply bridge all those different messaging systems together via adapters, then it sounds really appealing, doesn’t it? And SOA says so, too!
In a perfect world I’d say it might, but it opens the ‘can of worms’ which you would like to think about first.
And by the way those worms have names! They are called:

  • Governance & Processes
  • Team effort of IT & Business People
  • Life Cycle Management
  • Scalability
  • High Availability
  • etc

But the two biggest Challenges are less technical or implementation specific, they are of more social nature called

  • Ownership (most enterprises don’t have an Enterprise Architecture team) and
  • Cultural Change (Think Enterprise! instead of silo)

Hence a good portion of energy, time and money needs to be invested in bringing different  domain and silo owners together and they need to agree, too – all of them!

If they don’t, there will be no ‘Enterprise Service Bus’, not even as an architectural concept. And that is, because the whole Enterprise will not be connected through the same Bus and not all systems will be enabled to ‘magically work together’.

What’s your view?

Kindly,
Andreas Spanner