Integrating Islands with Landmasses

Choreography & Orchestration - Difference

2007-10-17T16:19:00.001+05:30

I have been having this doubt on the difference between Orchestration and Choreography for a while now. Learnt today that Choreography involves multiple parties while Orchestration focuses on a single one.

New Blog

2007-09-19T14:41:00.000+05:30

I've started posting at the Integration Consortium's blog. Do check it out at http://www.icmembers.org/blog/zaki

How to improve ROI with SOA?

2007-07-20T22:52:00.000+05:30

I was talking to a colleague of mine on using SOA in integration projects, when she cut through the chaff and asked me if we could show a better ROI when a project is done using SOA versus doing it "traditionally". Traditional means could be either by custom programming or buying commercial integration components/products, which would essentially be EAI. Most of the organizations are aware of EAI products and are engaged in doing integration using them in some form or the other, which leads us to the following questions:

Can doing an integration project with SOA get a significantly better ROI vs. doing it with EAI products?
In either case, do we need to optimize our business processes too to see a better ROI?

Any thoughts are welcome.

Few Minutes with Savvion Process Modeler

2007-04-21T14:14:00.000+05:30

I played around a while with Savvion Process Modeler and found the UI (non-eclipse) simple and intuitive. Modeling was straight forward and in my opinion the tool should have a flat learning curve and keep the Business Analyst interested. On the negative side, the tool does not give an option to save the model as a BPEL file, by default, it saves the model as a proprietary SPT file and can export only to an XML file. It has got pretty decent features for running the model as a simulation with Normal distribution, constant and exponential values that can be given to the variables.

For execution, the SPT file needs to be deployed onto the Savvion BPM server (BizLogic Server) for execution.

Overall, I liked the product for its speed and intuitive GUI but was not happy that it could not save the output as a BPEL file.

Actional SOAPStation - First Look

2007-03-04T17:47:00.000+05:30

Actional SOAPStation is a web services management tool with features for

Security
Failover
Monitoring
Reporting
Rule based routing

All the features are configurable through the administrative console. Installation is straight forward and the documentation is useful. Lack of examples make the learning a little steep.

It has extensive features for security, providing options such as SSL certificates, SAML token and WS-Security policies. Monitoring of web services helps check if a service is alive using Hearbeats. Configurable alerts can be sent through e-mail or a MOM. Actional SOAPStation also has a comprehensive reporting feature that enables it to generate reports based on date, events and other criteria. Routing of messages based on rules is possible. The rules inspect message contents and route the message to the appropriate service that can handle it.

In addition to the above features, Actional SOAPStation has features for generating dynamic WSDL based on the operations to be made visible. This enables it to function as a web services proxy.

In combination with a registry, this tool should help in providing governance of services across the organization.

Java - Now Open and Free

2006-11-13T21:40:00.000+05:30

Java is now Open and Free, things that a mean a lot from a community perspective - giving programmers who have benefited from the language a chance to give back to it by adding features/functionalities to it.

Also, it seems like the right time to go open, especially with several open-source Java projects doing well and more being announced. Sun gives the overview here.

Microsoft Dynamics AX (Axapta)

2006-10-25T20:45:00.000+05:30

When my cousin mentioned to me last week about Microsoft's new ERP product called Microsoft Dynamics AX, formerly known as Axapta, I got curious to know more about it and this is what I found:

MS Dynamics AX is mainly targeted at SMBs (Small & Medium Size Businesses) and focuses on manufacturing and the e-business sectors. The USP for this product, in my opinion, would be the ability to leverage existing MS resources in the organization such as Excel and MS-SQL Server. Since the product GUI's look and feel is consistent with other MS-Office products, it could result in a flatter learning curve. A language called X++, similar to C++ and Java, is used to program the product. MS has provided dotnet integration into the language as an attraction.

WPS and a Bunch of Specs From IBM

2006-09-15T11:37:00.000+05:30

I have been attending an IBM WebSphere Process Server training this week and here is the residue from what was taught.

WebSphere Process Server (WPS) is IBM's answer to the ESB, whoever has designed the product has probably not heard of the saying - "Make things as simple as possible but no simpler" because my assessment is that it is an extremely complex product. Nevertheless, it is a feature-filled product that provides capabilities on SOA and a whole new set of related specs. It has evolved from WebSphere InterChange Server (ICS) but transcends the EAI space as it uses WebSphere Application Server (WAS) as the underlying runtime. The components built on WPS are turned into J2EE modules and components during deployment. A significant benefit of using WAS is that WPS can be clustered, unlike ICS that had no clustering features by itself.

As with other IBM products, WPS too is resource hungry requiring 2 GB RAM for a development machine.

The Specs Arrive
In order to consolidate its position in the SOA area, IBM along with BEA has authored the following specs (except EMD and CEI which are IBM-only) and WPS implements all of them.

SCA: Service Component Architecture is a component model used to wrap components and expose them as Services. This is true for Java, WSDL and other WPS components. Other implementations of this spec are coming up, details are available at www.osoa.org.

SDO: Service Data Object is an API to access the elements in Business Objects (BO). Business Objects are data structures in WPS that carry data. They are implemented using XML schemas. The SDO spec does not mandate on how the Business Object is to be modeled.

EMD: Enterprise Metadata Discovery is an API to expose the meta-data of data structures such as database tables and XML schemas. This API is used by the WebSphere adapters to get the application details and create the WPS Business Objects.

CEI: Common Event Infrastructure is an IBM standard for monitoring. It provides a framework for capturing events and publishing them to different consumers. Events can be turned on/off for all WPS components.

Despite the negative points on the product's complexity, it should score on robustness as it is built on top of WAS and with IBM's aggressive selling and licensing policies (including giving away free licenses to existing ICS customers), there is going to be no stopping this product from getting a large customer base.

Micro and Macro Flows

2006-07-16T18:29:00.000+05:30

I have recently been hearing two new terms - "Micro Flow" and "Macro Flow". These are two fancy terms to describe the types of business processes.

Micro Flow means that a business process starts and ends in a single transaction whereas
Macro Flow means that the business process is long lived, requiring more than one transaction. Macro Flows are generally more complicated as they are composed of multiple transactions. The participating system's state cannot be changed in a single rollback and a compensating transaction or Compensation needs to be enforced. Workflows involving human approvals are examples of Macro Flows.

Learning Apache Axis - II

2006-06-15T23:20:00.000+05:30

I have been trying to understand how to develop a document style web service and it has been a frustrating experience to say the least. There is a lot of code required in the client, a number of interfaces to create, a number of utilities to run/re-run all for just sending an XML document instead of a string. I am fully aware of the benefits of this style but I am wondering if too much complexity would drive developers away from this approach to the more simpler RPC approach.

Learning Apache Axis - I

2006-06-12T17:49:00.002+05:30

In the past couple of weeks, I had been getting a lot of fundamental doubts in Web Services, so a friend asked me to learn Apache Axis to get an understanding on Web Services. So, I started with writing a simple web service and deploying it on Axis. Following are the steps to do so.

Create a webapp for axis in Tomcat or any other web container ($:\apache-tomcat-5.5.17\webapps\axis\WEB-INF\classes)
Create a service class with service methods
Create deploy.wsdd file (sample code pasted below)
Deploy using admin client - java org.apache.axis.client.AdminClient OrderDeploy.wsdd
Copy web service class to Tomcat classes folder - $:\apache-tomcat-5.5.17\webapps\axis\WEB-INF\classes
By default, the service will be deployed at - http://localhost:8080/axis/services/OrderService.
Invoke service at above location

Sample Web Service Deployment Descriptor (wsdd)
<deployment xmlns="http://xml.apache.org/axis/wsdd/"
xmlns:java="http://xml.apache.org/axis/wsdd/providers/java">
<service name="OrderService" provider="java:RPC">
<parameter name="className" value="OrderService"/>
<parameter name="allowedMethods" value="*"/>
</service>
</deployment>

I will provide more details in subsequent posts.

Data Federation

2006-05-25T02:58:00.000+05:30

Data Federation is a ETL term that means integrating data from multiple data sources. A federated system consists of :

+ A federated server such as DB2
+ A federated database
+ Remote datasets

Using a federated system, a single SQL statement can be used to fetch data from remote databases.

Distributed Development: the non-ICC approach

2006-05-24T01:46:00.000+05:30

From recent conversations with a fellow consultant, I learnt that the ICC approach had not sold well with a customer. I was surprised as I had always seen ICC as a successful concept. When I probed further for the reason, I came to know that the application groups in the concerned organization were well-staffed and skilled to handle their development and support needs. So they would be responsible for servicing their own needs. For governance on integration development, a Governance Committee would be created to define standards, best practices and guidelines for integration. This committee would also participate in higher level activities such as architecture definition and design reviews. But would not be involved in development.

So, this is an alternative approach to suggest when an organization's culture/environment does not favor an ICC.

Revisiting WMQ Basics - Client & Server Connection Channels

2006-03-26T23:52:00.000+05:30

Client and Server Connection Channels work in pairs to facilitate communication between the MQ clients and the server queue manager. The MQ client might exist on the same, or different, platform as the queue manager.

Server Connection Channel
The Server Connection Channel is defined on the server as follows:
DEFINE CHANNEL(CHAN1) CHLTYPE(SVRCONN) TRPTYPE(TCP) + DESCR('Server-connection to Client_1')

Client Connection Channel
The Client Connection Channel is defined on the client as follows:
SET MQSERVER=CHANNEL1/TCP/server-address(port)

Where:
* server-address is IP address/machine name of the server
* (port) is the port number the server is listening on

RosettaNet Overview

2006-03-23T10:13:00.000+05:30

I have been wanting to read up on RosettaNet for a long time now and finally managed to do so. In my earlier visits to the the site, the site design had dissuaded me from exploring it further. But now, the site appears to have been revamped and navigation is simple and easy.

RosettaNet standards can be used in B2B communication among trading partners similar to ANSI EDI X12. The standards are defined in terms of Partner Interface Processes (PIP). Partner Interface Processes define the rules for communication (sequence of messages, timeframe etc) and the format of the message used in communication. The message format is enforced through an XML DTD. PIPs exist for 7 functional areas called clusters. Order Management, Product Information and Inventory Management are examples of some clusters.

Purchase Order DTD part of the Purchase Order PIP

Revisiting WMQ Basics - Queues & Channels

2006-03-19T22:29:00.000+05:30

I was informed of an assignment related to WebSphere MQ troubleshooting and performance enhancement but had to turn it down as I have been off MQ for quite sometime. However, I've decided now that I will refresh my MQ skills and become hands-on. I am starting from the basics and will post interesting/important notes here as I go along. I may choose to skip topics that seem a bit too obvious to me.

Queues
WebSphere MQ has four types of queues: Local, Remote, Alias and Model queue. The first 3 function as their names' suggest, the last one - Model queue is a template that can be used to define new queues with the same properties.

Triggering
Triggering happens when a message appears on an Initiation queue. The process defined and associated with the trigger is started when a message with the matching trigger critieria occurs on the queue.

Message Channels
Message Channels communicate between Queue Managers and can transmit messages in one direction only.

A transmission queue (XMITQ), which is a local queue is mentioned in the channel definition to associate it with the channel.

Types of Channels

Sender: Sends message and initiates connection to Receiver
Receiver: Waits for initiation from Sender
Cluster-Sender: Used in Cluster Queue Managers (need to find out more)
Cluster-Receiver: Used in Cluster Queue Managers (need to find out more)
Requester: Sends request and becomes receiver
Server: Accepts request to start from requester, then becomes sender

Two channel types that I need to look at are Client Connection and Server Connection.

Channels are initiated through programs called Message Channel Agents (MCA). Persistent messaging is handled by MCAs as they are the ones who deliver the messages. The message is not removed from the xmit queue (transmission queue) until the MCA confirms the placement on the target queue.

I will next take a look at the properties of MQ messages.

XML Design Patterns

2006-03-09T17:21:00.000+05:30

I was reading a document on Service Payload design in SOA and stumbled upon this page on design patterns in XML by Kyle Downey. It talks about four patterns -

Dynamic Document
Composition
Self-Documenting Files
Multipart Files

Use of relevant examples in the site makes it a good place to start with when designing XML messages/structures.

SAP Integration Basics

2006-02-23T14:55:00.000+05:30

There are two ways of integrating external systems with SAP R/3.

ALE
BAPI

ALE - Application Link Enabling, part of SAP's integration layer enables integration between SAP R/3 systems and other external systems. It is best suitable for use with batch objects and asynchronous integration.

BAPI - Business APIs are SAP's APIs to interact with SAP R/3. They are implemented as RFC functions for an SAP business object's (e.g. Customer, Vendor etc) methods. BAPI is best suitable for real time synchronous integration where the BAPI method is invoked as a request and the resultant data is returned as a reply (request-reply paradigm).

SOA Startup?

2006-01-10T23:41:00.000+05:30

After reading David Linthicum's post on SOA startups being acquired early, I was wondering whether I should quit my job and start a startup! Maybe a company that builds an ESB that runs on mobile devices? (Is it a worthwhile idea?) I would go to Fred Wilson of Union Square Ventures for VC funding, hire J2ME and Pocket C programmers and start rolling. I will also generate hype through various means and one day somebody will acquire the startup, I will be made the Vice-President and live happily ever after!!

Microsoft in EAI

2006-01-01T01:09:00.000+05:30

My knowledge of EAI is largely concentrated with IBM, Java and Open Source middleware. This week, I completed a course on IBM WBI Modeler and will post a summary on it shortly. After that I plan to spend sometime learning Microsoft's offerings in EAI, notably BizTalk.

Now an Integration Appliance!

2005-12-31T11:43:00.000+05:30

I was reading literature on David Linthicum's Bridgewerx appliance for SMBs (Small and Medium Scale Businesses) and also attended a webinar that touched upon the product. It offers out of the box integration through:

Integration software hosted at the Bridgewerx site
Appliance acting as the hardware

The first step is to design the integration at the Bridgewerx site. The integration software is a Macromedia based web application and is available for a test run at the Bridgewerx test drive section. The output of the design task is a file that has to be configured and deployed on the appliance.

After designing the integration at the Bridgewerx site, the endpoints (source and target) have to be configured and the integration can then be deployed on the appliance. For each environment e.g. Test, Production, a separate appliance would be required.

The Bridgewerx appliance is a Dell Wintel server running Windows 2003 Server, BizTalk and SQL Server 2000. The complete specs are available at the Bridgewerx site. The choice of a Microsoft based solution instead of an Open Source approach has been a little intriguing to me. But then, this could be due to the fact that most of my experience is with IBM and Java based middleware.

In the webinar, a solution using the Bridgewerx appliance was explained. The scenario was integration of Salesforce.com with an internal billing application of Bridgewerx's client.

DataPower is another appliances vendor which has a similar product - XI50. I could not get much details on its specifications, so will not be able to compare it with Bridgewerx's product.

Bridgewerx is clearly aiming at the cost-conscious SMB market with an extremely competitive pricing strategy. With IBM acquiring DataPower, it would be interesting to see how the competition in this market unfolds.

Are ICCs really successful?

2005-12-18T22:03:00.000+05:30

Amidst widespread hype about the success of Integration Competency Centers (ICC), I have been hearing rumours of the ICC proving to be costly in at least two of the clients with whom I was/am associated with. Aside from cost (inspite of offshoring), the other reason that is spoken against the ICC is inflexibility in terms of accepting and executing requirements. This reason in most cases is true and could be explained as follows:

Choice of Integration Tool
These days the EAI tools predominantly focus on realtime transactions and process integration. Many organizations think that they are ready for realtime process integration while a deeper insight into their functioning show that they are not. This is because most of their business processes are designed to be batch oriented and changing them would have a huge impact with adverse effects. Most of the EAI requirements center around data integration levels and only require the tool to move data across systems. Thus the EAI tool in most cases, is not fully effective.

ERP Solution Centers and Process Integration
The next problem is that when an EAI tool, whose general definition is to do data integration tries to get into ERP space for process integration, it is definitely frowned upon. The ERP solution centers that manage business processes as a core functionality see this as an invasion into their space and friction is bound to develop.

Scarcity of Skilled EAI Resources
The problem with getting EAI skilled analysts and developers is bigger than ever before with more and more companies waking up to EAI. A fallout of this is that resources act high-handedly, dictating terms to prospective employers and demanding exorbitant salaries. And surprisingly, most of them are folks with just an year or so on a single EAI tool. More often than not, they fair poorly on the job as they are more of "button pushers" or "machine operators" with hardly any knowledge of EAI concepts and paradigms.

So, where does that leave us?
It may so happen that there are no concentrated competency centers doing integration anymore. Folks on a project could just double up as EAI developers as per the project requirement. This would serve to alleviate the resourcing problem and a more informed decision on the integration tool could be made as the project resources would be the ones closest to the functional requirements.

IASA President visits this blog!

2005-12-08T23:36:00.000+05:30

I was delighted to find IASA president Paul Preiss's comments on this post. It is encouraging when people read a blog and leave comments, the joy is doubled when they happen to be dignitaries.

Efficient XML Interchange - EXI

2005-12-05T23:38:00.000+05:30

In almost all of the EAI projects that I've worked with, I have noticed one problem related to performance - the growth of message size when wrapped in XML. This is of concern especially when the data being transferred has no semantic meaning, binary data, database records being synchronized and similar examples come to mind. People have resorted to actions such as removing the XML wrapping, using EDI internally(!), switching to FTP for data transfer and even upgrading hardware servers (due to stubborn EAI evangelists with substantial clout) to overcome this. The affected parties can now take heart from the fact that the W3C has setup a working group called EXI to develop a format that improves the performance while transferring data in XML. (news through David Linthicum's Real World SOA blog). Of specific interest to me is the following point in the charter,

"Fulfill the design goals of XML with the following exceptions:

The interchange format must be compatible with the XML Information Set instead of being “compatible with SGML” (XML goal 3);
For performance reasons, the format is not required to be “human–legible and reasonably clear” (XML goal 6); ..."

Point 1 related to XML goal 3 will hopefully shed off some baggage that the present XML Info set is being burdened with. Point 2 should be the differentiator for this format and deal the killer blow. Not sure how this point will be taken care though, with entities probably?

SAP Internet Sales (R/3) Overview

2005-11-28T10:20:00.000+05:30

I am presently involved in an integration with the relatively new SAP Internet Sales product. Here is an overview on it.

SAP Internet Sales (R/3) is part of the mySAP CRM suite based on the Self Service pattern. It is an Order Management and CRM System that allows customers to browse products, configure and place orders.

Architecture overview
Internally, SAP Internet Sales uses Material Management, Customer Master, Catalog Management, Order Management and Contract Management modules. It is based on an MVC architecture on top of the SAP J2EE engine. Layers are present that map to the Model, View and Control as follows:

Presentation/View: Interaction and Presentation Layer
Model: Business Objects Layer
Control: Business Logic Services Layer

The communication with the model happens through a Java Connector. An Index Server to optimize search/browse of products and a Pricing & Configurator module are built in.

Other features in SAP Internet Sales are:

Orders are saved in real time in R/3 with no delay or intermediate storage
Web access user accounts are created for customers existing in SAP R/3. The usual process of users registering through the web for accounts in R/3 is also possible.
Material Master module is the foundation of the catalog
Product catalog uses the material data, prices, multimedia objects and text descriptions to build the catalog

Estimating Development Effort in EAI - II

2005-11-21T13:56:00.000+05:30

Following Vivek and Controlled Agility's thoughts on my post on Estimation in EAI projects, I learnt Function Point (FP) estimation and realized that the thoughts on my earlier post do not hold true for estimation during all phases. FP is the technique to use when estimating at an earlier phase, say costing or discovery, as there are very few details available. Work Breakdown Structure (WBS), on the other hand, is best suited for estimation in the development phase, when all the details are available, the development tool has been finalized and resources with expertise in the tool are also available.

I had plans to rewrite my earlier post with FP as the base, but will not do so now because Controlled Agility covers it in more depth in this post.

IBM, make up your mind!

2005-11-12T12:24:00.000+05:30

I was deciding an integration architecture for a prospective client and suggested the use of IBM Interchange Server (ICS) for an A2A requirement. Another colleague who was documenting the approach mentioned that ICS was not sold independently now and offered as part of WebSphere Integration Server and showed the following links as evidence - http://www-306.ibm.com/software/integration/wbiserver/ and http://www-306.ibm.com/software/integration/wbiserver/faq-43.html. So, we decided to follow the nomenclature and used the name WebSphere Integration Server in our architecture.

When we checked the price of the products with IBM, their representative mentioned that ICS was no longer part of WebSphere Integration Server and was part of another product called WebSphere Server Foundation, so we went ahead and updated the product name. After a short while another IBM representative spoke with us to clarify a few technical queries and mentioned that ICS was part of WebSphere Integration Server and not Server Foundation!

When the final architecture was reviewed, a colleague presented an IBM announcement mail in which it was mentioned that WebSphere Integration Server was being sunset and replaced with ICS independently! At this point, we were at our wit's end. As of the time of this post, I am still not sure whether ICS exists as an independent or as part of WebSphere Integration Server or WebSphere Server Foundation. IBM may make great products but their naming and packaging leaves a lot to be desired.

Point Solutions

2005-10-28T22:48:00.000+05:30

I was looking at understanding what a Point Solution meant and came across this post by Fred Wilson, a VC. From his post, I understood that a Point Solution was aimed at addressing a specific problem rather than acting as an end to end solution. Google is a classic example of a point solution providing search capabilities. Though it provides other services, it is not integrated like Yahoo! is. To elaborate further, though Blogger is part of Google, the registration for using Blogger is independent of the registration required to use GMail, whereas Yahoo! is a portal, offering services customized for each user.

While many people like the concept of a portal as an end to end solution, there are others that would only like to take only how much they can chew. As Fred notes in his post, web sites with simple independent services such as del.icio.us that offer bookmark management are point solutions that are very popular.

IASA

2005-10-20T19:40:00.000+05:30

I stumbled across the website of IASA - International Association of Software Architects. One of their primary tasks is standardizing the role of a Software Architect and they are compiling a taxonomy to that effect. Initially, I did not think much about it but when I read that Grady Booch was on their board of advisors, I went ahead and registered. Their website is in beta and registration is a pain, I had to enter the confirmation code several times before getting myself successfully registered. But it is still a good idea to register yourself as there seem to be a quite a few things planned.

Integration Choices: Interactive vs. Non-Interactive

2005-10-15T13:49:00.000+05:30

A colleague of mine had a question on when a user interface (UI) would need to be provided to invoke a business service vs. using a database trigger to do the same. According to me,

When the business pattern is Self Service i.e. the user wants to query the status of an order or place a claim (interactive), a UI would be most appropriate. In other words, whenever there is an interaction between a user and a system/application.
When the business pattern is Application Integration i.e. synchronization between multiple databases or legacy systems or batch driven updates (non-interactive), a database trigger could be chosen. In simpler terms, whenever there is interaction among applications/systems rather than users and applications.

SOA - Can we see some real projects please?

2005-10-13T13:28:00.000+05:30

It is been over 4 months since my first brush with SOA. I am now beginning to wonder if there are any real SOA projects being implemented at all. By real SOA projects, I do not mean pilots or proof of concepts (POC) or projects that have been existing pre-SOA and rebranded SOA compliant with a few Web Services thrown in. Rather I am looking at projects that have been started from scratch with SOA as the base. Unfortunately, I have not seen any real world implementations of SOA and have begun to doubt if there will ever be a SOA project.

Virtualization

2005-09-17T22:39:00.000+05:30

I have been involved in the setup of one of the servers and was doing some research on Virtualization. Virtualization is the process of abstracting hardware details from the software that runs on top of it. VMWare is one of the popular software used for virtualization, IBM too has software that does the same thing. Two operating systems could be made to run on top of a virtual layer created by VMWare running on the server, which would make the OS believe that there are two servers available.

One of the common uses of hardware virtualization are for sharing a server between two environments. I am not sure if it helps in reducing product licenses.

Estimating Interface Development Effort in EAI Projects

2005-08-28T16:50:00.000+05:30

Recently, I had been assigned the task of reviewing and improving the effort estimation in developing interfaces in EAI projects. I had tried this in the past but with little success.

To understand the problems in estimating EAI projects, we would need to understand how EAI projects are implemented. The following two approaches are largely followed in the development of EAI projects:

Directly using Off the Shelf EAI products
Building and using frameworks on top of EAI products

When developing directly using EAI products, the problems are that each product comes with its own unique set of components to define an interface. For instance, an interface in IBM's WBI InterChange Server would largely be built using

Connectors
Business Objects
Maps
Collaborations

where as in a project that uses Mercator (now WebSphere DataStage TX) and IBM WebSphere MQ, the interface would be composed of

Adapters
Type Trees
Maps

We could roughly equate Adapters with Connectors, Business Objects with Type Trees and Maps with Maps at an architecture level. Collaborations, though are absent in Mercator. However, at an implementation level, the differences are too much to be left unconsidered. For instance, Business Objects are totally based on XML while Type Trees are not. Maps in ICS allow for extensive custom code to be written while in Mercator, only limited custom code can be written using the in-built scripting language. The absence of a component (Collaboration) also makes a great deal of difference. These are scenarios using only two of the many products available in the market. When we consider all available EAI products it becomes uncomprehensible.

The framework approach involves building a layer over the EAI products in order to add commonly required features and reduce development effort. The aforementioned problem applies to this situation too, as there is no standard way of defining an interface leading to frameworks that are specific to EAI products.

So, in order to arrive at a standard way of estimating EAI interface efforts, the product vendors need to define interfaces in a standard way. They should decide and agree that an interface would consist of standard components, for example, Adapters, Message Sets, Maps and Message Flows that would in turn be built of standard units, e.g rules, parameters etc The competion among the vendors would then be on providing the best components and the best server. Till then, interfaces would remain kludges and estimating their development time, unreliable.

Software Architect Bootcamp Book Notes

2005-08-11T00:18:00.000+05:30

I was browsing through the book Software Architect Bootcamp by Malveau and Mowbray and noted down a few interesting points.

Rule of Three

A single design occurrence is an event
Two design occurrences is a coincidence
Three design occurrences is a pattern

Classic Reference Model for Consulting Intervention

What is the problem?
What are others doing to contribute to the problem?
What are you doing to contribute to the problem?

Components Vs. Objects - Principles of Components

Encapsulation
Polymorphism
Late binding (composition during design time)
Safety (e.g. in Java, memory safety is guaranteed through Garbage Collection)
No inheritance - delegation (invocation) is used instead

Publish Subscribe with Ack

2005-08-02T22:35:00.000+05:30

In a past project, we had implemented a "new EAI pattern" - Publish Subscribe with acknowledgement. How this was different from the regular Publish Subscribe was that all the subscribers would acknowledge the receipt of the message to the publisher, adding another layer to "guaranteed" delivery - a cross between Request Reply and Publish Subscribe patterns. The usage scenario was, while sending master data from a Source of Record to all subscribing systems, the receipt of the data was to be ensured as it was functionally critical.

I am not too convinced that this is indeed a new pattern as publishers in the Publish Subscribe pattern, need not be aware of the subscribers. But in this case they would have to be.

Short talk on Canonical Messaging

2005-07-18T22:34:00.000+05:30

I gave a short talk on Canonical Messaging to colleagues I work with. It went well and I felt good to be back in front of an audience after a gap. The contents of the talk where derived largely from the white paper that I wrote a long time ago. The white paper is available at the link mentioned in this post's title.

Facade to the rescue

2005-07-13T22:40:00.000+05:30

Whenever I have been at a loss while explaining on Service enabling of existing applications (bringing them into the SOA fold) , I have always hidden myself behind the Facade pattern - all details are abstracted from the invoking application and only the service to be invoked is exposed!

BPEL in a nutshell

2005-06-27T00:45:00.000+05:30

BPEL (Business Process Execution Language) is a language that can be used to model business process flows. BPELScript is an XML schema based script that is used to write or generate process flow steps (activities). A runtime interpretor aka BPEL Engine executes the BPEL scripts. A Partner Link contains the context or role of the BPEL script and the application(s) interactions. Partner Links are present as tags in the BPEL script. Client and Service Provider (Server) are a few roles that can be defined using a Partner Link.

Anatomy of a BPEL Script
A BPEL script, in general, consists of:

A root tag named Process
Required WSDL or other schema imports
Partner Links to establish the role of this BPEL script (process) and the interacting applications
Variable definitions
Fault Handlers and Event Handlers
A sequence containing any of the activities defined below.

Activities
Process modeling in BPEL is done using activities and there are four main types of activities in BPEL.

Communication: These activities are for receiving input messages or text from other applications or invoking methods in other applications to get input.
Control: Control activities are the control structures in BPEL. Switch, Case, Otherwise, While, Flow and Wait are the activities available.
Fault: Fault activities are used to handle errors and exceptions. Throw, Catch, CatchAll, Terminate and Compensate are the activities available.
Other: Other activities are available for assigning, copying or transforming variables and returning their values

In order to better understand the capabilities of BPEL, I hope to model a process using the different Partner Links and activities available in BPEL. I will do a post on the outcome of that activity here. Lack of time led to the Cape Clear license getting expired, so no more BPEL :(

SOA - So what's new?

2005-06-23T22:30:00.000+05:30

I was talking to a colleague on SOA, ESB and other related stuff that I've blogged here. He gave me a patient hearing and asked, "so what's new?" He said that all this SOA stuff was already being done in the EAI space for a long time now, even before the term SOA was coined. "An adapter is a service, right?", he asked and I said yes and went back to square one - asking myself, so what is an SOA? More importantly, what is a "service"?

[Update]
After posting on my state of mind above, I went ahead Googling for SOA definitions and reasons, hoping to find better meaning and purpose for its existence. I found John Reynolds' post "The SOA Elevator Speech" very satisfying. The perspective on SOA is that while normal integration focusses on the technical aspects, SOA does that at a business level - all services are described at a business level rather than a technical level (to paraphrase John: processing an order is a service in a SOA while updating a record in a database is not).

I hope that I am now better prepared to take on that colleague of mine.

Routing in Cape Clear 6

2005-06-18T00:34:00.000+05:30

According to Gartner, an ESB should provide the following features:

Transportation
Routing
Transformation
Web Services

Let us examine the Routing features in Cape Clear 6. Routing in Cape Clear 6 can be setup in the Cape Clear Manager by adding "Routes". Routes are made up of "destination endpoints" that could either be HTTP/HTTPS URLs or JMS (JBoss/IBM MQ) destinations. Complex routing (similar to having a hub/router) can be setup as a composition of multiple routes.

Content based routing is accomplished using a XPath or a regular expression match. The messages are searched for the expression and routed to the web service subscribing to that expression. The expressions are set at the "Route" level.

The source for an integration is always a Web Service where as the destinations could be HTTP URLs or JMS destinations (topic or a queue) endpoint, as mentioned previously.

Cape Clear 6 provides a service for transformation to/from multiple formats including Excel, Flat files, HTTP Post, Custom Java and XML. XSLT is used to develop the mappings.

Buy vs. Build Strategies

2005-06-15T21:57:00.000+05:30

I have always been in favour of the "Buy" approach for integration projects rather than "Build" considering the many advantages that it brings. Ronan Bradley, CEO of Polar Lake has written a good blog entry at the Integration Consortium Soapbox. Do take a look at it, it makes interesting reading.

Hello Cape Clear 6!

2005-06-12T01:08:00.000+05:30

I was playing around with Cape Clear 6, an Enterprise Service Bus (ESB), to understand web services better and to get to know the concepts of an ESB. My first impression was that it was remarkably easy to understand and use. Though I only went through the Hello World tutorial, I thought the concepts of Design, Develop, Deploy and Test were mapped clearly to the product GUI. Following are the points that I learnt from the Hello World tutorial on building web services. The corresponding phases in the Web Services Development cycle are mentioned within brackets in bold.

Generate a WSDL (Design)
Add operations in the WSDL for accepting request and sending a response (Design)
Generate Java server code (Develop)
Edit the server code to add business logic corresponding to step 2 (Develop)
Rebuild the server jar (Develop)
Deploy the web service (includes packaging the server jar file into a WSAR) (Deploy)
Generate and deploy a JSP web client for testing the web service (Deploy)
Test the web service through the JSP web client (Test)

The next task is to actually build a BPEL based Orchestration Service and use Cape Clear 6 as an ESB. BPEL - Business Process Execution Language based on WSDL, would probably replace Java in steps 3 and 4 mentioned above. An XML based language that can actually be executed, should be fun!

IBM's SOA seminar - A Construal

2005-06-09T23:21:00.000+05:30

I had attended a seminar by IBM on Service Oriented Architecture (SOA) sometime last month and thought that I should share my comprehension on the points that I thought were interesting:

IBM mentioned that SOA was more suitable for migrating existing architectures rather than building new applications from scratch
Web Services Gateway can be used for routing; as an interface between a web services client and a legacy application that was being web enabled. Enterprise Service Bus (ESB) can be used as a broker and it appears to be a successor to the broker architecture.
An overview of BPEL4WS (Business Process Execution Language for Web Services) was provided. It is a language for business process modeling similar to collaborations in IBM WBI Interchange Server.
The site http://www.ws-i.org details the Web Services interoperability standards.
For transporting large volume of data, JMS queues could be used instead of HTTP for reliable delivery
Security aspects related to Web Services were touched upon and it was mentioned that standards present for HTTPS would possibly be adopted for Web Services
New tags such as would be present in the SOAP header to indicate if the SOAP envelope was signed
SOA Design Patterns and Anti-Patterns were discussed. Following are the details:

Patterns

Unified Logical View: Facade pattern - presents one service to the external world and routes the request internally to the appropriate subsystem
Adapters for legacy code - This is similar to the EAI adapters. Web services are exposed as adapters.
Composable components (BPEL) - Using BPEL, each process could be modeled as a task
Replaceable components - Develop components that are implementation independent

Anti-Patterns

SOAP over HTTP or XML should not be used within layers of an application
It should be used only when exposing a layer of the application to other applications

I would like to add a general disclaimer that the above points are according to my understanding, it is possible that I might have misinterpreted some of the points (but it is highly unlikely :)).

MQ 6.0 Open Beta Available

2005-06-06T00:35:00.000+05:30

Looks like the IBM WebSphere MQ Open beta v6.0 has been available for a while now. I am excited to learn that there would be integrated support for web services, which probably means that there would be some intelligence built into the product to understand SOAP requests. Another feature that caught my eye was that the new administrative GUI for the product is built on the Eclipse platform. ~~I will hopefully download the product soon and try these new features out. Stay tuned for a review.~~

Update: Laziness and other higher priority tasks have made this activity a non-starter.

Integration Architect? Who's that?

2005-06-03T23:16:00.000+05:30

I've searched the web a good deal to find out an Integration Architect's role and responsibilities, and found quite a few definitions. But most of them seem to be specific to a requirement, so I thought I should create one that is more generic. According to me, an Integration Architect would be the one who defines the architecture for integrating applications (using a tool or a framework) and creates a methodology for implementing "Integrations" or "Interfaces" between applications using the defined architecture.

He/she would typically go through the standard development lifecycle from inception to delivery. The involvement from the architect's perspective would be heavy during inception, light in the middle and moderate towards the end. Tasks the architect would perform in the large middle period would be guiding the implementation team and towards the end would largely be validating the solution against the requirements.