Tuesday, 29 January 2013

Architectural design for federated Corporate Credit Commercial Capital Market Banking Applications using Service Oriented Architecture (SOA) and Enterprise Service Bus - Sandeep Kanao

Architectural design for federated Corporate Credit Commercial Capital Market Banking Applications using Service Oriented Architecture (SOA) and Enterprise Service Bus - Sandeep Kanao


Goal :
One of the major banks in Canada has  several existing corporate commercial applications (CCL) in ASP and ASP.NET.  With the bank mergers, new application was initiated to cater need for small business (< 50 M). This application needs information from the existing CCL application suits.
Since all these  corporate commercial applications (CCL) are stable and hosted/maintained by several different departments, we cannot modify or make any changes. The goal of the new project is to develop federated security services, so user can be authenticated on all the existing CCL web applications using the same token and consume the required services. This is achievable by service oriented architecture (SOA) and enterprise service bus. Application must supports dynamic resolution of endpoint (Intelligent Routing) for any request by the new application.

Interoperability needs
The 'services' look to be more like asp pages that provide access to functionality in an API-like fashion. What is lost is the uniform programmatic way to access these 'APIs' (since each page could come with its own specific way to interact with it). We can build some adaptors that hide the particularities (wrap the calls to the ASP pages in web services. That way we can hide the call to the ASP page and the parameter passing behind a nicer programmatic interface).

Data integrity needs
Here we potentially have a BIG problem, provided that we are dealing with distributed transactions against multiple resource managers/databases. In very practical terms, we have to ask our self how to rollback changes when our APIs come in the form of ASP pages, that obviously cannot enlist in any distributed transaction.
If  distributed transactions is not needed, we can create another integration level with different apps (as for example at the middle tier level or database level). Some of these 'services' may not  allow to reuse the functionality.


Application layer uses :
Neudesic-Neuron, WCF4, MSMQ Appfabric to supports dynamic resolution of endpoints (Intelligent Routing)

Friday, 11 January 2013

In Memory Cloud Datagrid Technologies - Capital Market VaR generation - Sandeep Kanao

In Memory Cloud Datagrid Technologies - Capital Market VaR generation - Sandeep  Kanao

The move to in-memory is all about achieving the best performance, by accessing data held in a server's random access memory (RAM) as opposed to on a hard disk.  Typical access speed for RAM is 0.4 nanoseconds, whereas for disk it's 4 milliseconds - so RAM is 10 million times faster.
A number of in-memory cloud data grid offerings are available, including:
  •  ActiveSpaces from Tibco Software
  • Oracle's Coherence
  • Armanta's Intelligence Services
  • GemFire from VMware
  • Quartet FS's ActivePivot in-memory analytics
  • SAP's High Performance Analytic Appliance (HANA)
  • ScaleOut Software's StateServer
  • GigaSpaces Technologies XAP platform
  •  BigMemory from Software AG's Terracotta unit
  • Kognitio's In-Memory Analytics Platform
  • GridGain's In-Memory Compute and Data Grid
  • Open source Memcached
  • NCache (.NET)
  • Microsoft Appfabric (.NET)
In Memory Cloud Datagrid Technologies POC application background:

We wanted to try in-memory database for the VaR simulation. VaR engine is hosted on Solaris (2 T44) Solaris boxes with 128 cores each. Application is written in C/C++ (64 bit) and database is Sybase. VaR simulation runs takes 8 hours. The goal is to support at least three VaR within the stipulated window (8 hours) along with performance improvements in the current logic.

We have identified the performance bottleneck within the application. It is found that VaR simulation as well as scenario read/write to Sybase database takes maximum amount of time (over 30%). As a result we evaluated following three in-memory datbases :
  •  Activespaces from tibco (installed on the same rack as the client) on 2 - T44 Solaris boxes with 64GB  memory each
  • GemFire from VmWare (installed on the same rack as the client) on 2 - T44 Solaris boxes with 64GB om memory each
  • SAP Hana, installed on the data grid - 3 - T44 Solaris boxes with 64GB memory each
I will put the results of above  In Memory Cloud Datagrid Technologies PoC in the next post.