The Data Refinery

Two years ago I was involved in a RFID project in the logistics market in joint Capgemini and Oracle effort. Based upon the EPCIS Architecture we had to combine movement data of crates between different storage facilities and stores. Without going into details about the case, on the technology side the biggest challenge in RFID is in the huge amount of data and how to get data out which makes sense for a particular use case. We came up with a division of the life cycle of RFID data in three main segments (in discussion with Ard Jan Vethman) (1) In real time (within seconds), you want direct actions, a crate is entering a room where it is not supposed/expected to. (2) In short term (hours, days, weeks, depending on the business case), you want to perform actions based upon a condition based upon multiple measurements. for instance when a shipment of fresh food between a factory and a warehouse normally takes 4 hours, you want to send an an alert when the shipment takes a day. (3) In longer term (more then weeks), you want to see how your processes are optimized against the measurement which are received. You can look at this as a Data Refinery, bulk data is processed as it flows in (complex event processing), the result or residue is a high quality small subset of all the data which is entered, the exceptions from all that is ok. The residue is then used as events which can be send to a Business administrator and systems administrator. Oracle provides the following technologies which can be used for this functionality. For the longer term we've known BI for a long time. But what is provided in the real/short term. For the short term data Oracle Business Activity Monitoring (BAM) has now been around for some years, and provides excellent insight, as well as graphically as with alerts. For the real time processing we had to program the event processing ourselves, but recently Oracle came up with Complex Event Processing (CEP) as a COTS product. This enables querying data on a data stream. Getting insight in a near real time data stream and do some cherry picking is still a new and under developed area and is an exciting area.

Aim for Stability Based Architecture (SBA)

Software Architecture is the art of translating Business needs into a suitable and working technical solution.
The same as in architecting and creating houses our sponsor expects that the product will be stable and last for quiete a time. Looking back in time we see that at the time of Cobol, applications were able to last more then 20 years (and even now exist, even after the retirement of their creators). But with the speed of new developments and the coming of hype driven development the application lifespan (or time that it gets outdated), is getting less and less. Java, among others, has got a bad reputation in this area, for instance Struts lasted some 2 to 3 years, after which you were the laugh of the year when you decided to choose Struts in your solution. SOA is another example, finally it paid of working with it, when somebody decided that SOA is dead and we have to run after the latest hype, Cloud computing. The half-time of a Software Architecture is getting shorter and shorter, and it happens too often that at the time of go-live we almost have to think about upgrading, because the version is almost outdated.
I'd like to propose a new architecture type, Stability Based Architecture (SBA), in which we aim for stable (core of the) application and add an expected life-time of the application. The core of the application should be stable and last for a long time, just like rock under pressure, it folds but doesn't break. Only then can we justify the costs of implementation.

The Return of Oracle Forms

The Return of Oracle Forms

Oracle Forms has been around now for some 20 years, it looked like to get extinct over time, but the financial crisis might give it second chance.
Oracle Forms is a GUI used typically in transaction based applications for high volume data entry. Forms started of as a character based application in the 80ies, it evolved into a GUI client in the early 90ies and now can be deployed as a applet based application used within a browser. The basis of Oracle Forms hasn't changed much. A dedicated Forms process is connected to a Database. In the early days this Forms process was tightly connected to the GUI. In the current implementation the Form process is located on the Application Server and the GUI is located in a browser on the client. Big difference with a 'normal' Web Application is that the connection between the Forms Client and the Application Server is stateful whereas with a a 'normal' Web Application it is stateless. This articulates the type of usage, for every user a separate Forms Process will be started on the server, which consumes processing power and memory. In short, Forms is typically used by a known community, a Web Application can be used by an unknown community. When looking at application types, Oracle Forms is excellent for high volume data entry, and aligns with non functionals like minimal mouse usage etc. In terms of development effort Oracle Forms still is way more productive then any Java based application (roughly 4 hours per function point versus 8 hours).
A couple of years ago the statement was that Oracle Forms is outdated, but slowly it gets back in the picture of Oracle.
In The Netherlands we still have a large community of clients with (core) applications working on Oracle Forms (and not the latest version.....). So for those clients talking about SOA is shouting from over the horizon. And certainly in this financial climate new investments are hard to get.

Currently the step towards SOA is too big (and expensive) for organizations, but revitalization of the core systems build based upon Forms will be a must the coming years. Forms can be used as stepping stone towards a SOA based architecture. Forms can nowadays be used as a channel service exposed in a Web Portal(see also ADF Forms) and also can call out to (Web)services. And Oracle is investing again in Forms Forms modernization.

So is Oracle Forms going to get a second chance?

Desiging XML, reuse is nice but aim at usage first!

Designing Data has been, for ages, the cornerstone for all applications. It relates to all information needed and shared between the applications. This is no different when we look at the SOA world. Actually it starts to get even more important. First of all, for what data is a Service responsible? And secondly what information is shared between the Services.
The first part is all about the definition of a data model, as (mostly) used in a database. This definition is used for storing the information and assuring validity. The second part is about the definition of XML documents, as used in the definition of Services boundaries, this data is more fluid floating around between services.

Now what makes a good Data Designer? In the Database world it took us a while to learn how to define a thorough data model. It is not only about WHAT data is needed but also HOW the data is used. What type of queries are to be expected. What are the quantities of the data per functionality. How is the data related to one another, etc. A lot of 'old fashioned' techniques, like NIAM, helped is us in the past in defining a good data model.

When I look at the XML world, to my surprise, a lot of the 'good old designing tricks' are forgotten. In projects I come across XML definitions (coming from standard organizations), which are bound to fail in practice. Yes, all the data needed for the functionality is there (the WHAT question). The HOW question, however, most of the times, is completely forgotten. In theory it looks very beautiful creating an XML tree containing all functionality needed in the world. In practice this means, most of the time, that the ratio 'actual needed data' to 'Total data send' is less the 5%. This puts a burden on performance and network load.
It is excellent that one XML Document can contain all the information needed to share across all the services, but in practice this means that when you change one part of it, all services need to be changed.

With the ever increasing need for exchanging data it becomes essential that we should aim for a methodology for designing XML documents. In the meantime, let's first use the KISS (Keep it Simple Stupid) principle, and see if the XML design really works in practice, on both a functional and usage level. Only after that start looking at reuse.....

Plea for standardization on consumer good chargers!

Once a month I do some travelling. Whether it be for business or fun I usually carry around my gear I need, laptop(s), phone, eBook reader, IPod, camera etc. All these appliances need to be charged once so may days. No problem with that, but what annoys me is that EVERY appliance needs a DIFFERENT charger in different type, size and weight. It gets even worse, there is no charger that I can reuse! And when I travel from country to country I also need adapters for power plugs, and even sometimes, adapters for adapters. The whole IT world has been buzzing about SOA talk (Service Oriented Architecture) with the basis that everything should be based upon Open Standards. But when you look at the basic infrastructure for consumer goods, every brand delivers it's own type of charger. What is the problem with defining a standard for charging consumer goods? Let's apply the SOA principles also here! It will save me (and all other travelers around the world) from carrying around a surplus of wires, adapters and chargers.

Thanks in advance!

Reduce our IT CO2 foot print, let's green label our applications!

After reading the book cradle to cradle I started thinking what we can do in IT about reducing our C02 footprint. People start to realize that IT is consuming some 5% of our energy consumption nowadays. So all hardware companies start thinking about how to improve the energy consumption of servers. This improvement is only aimed at the run time component of the application execution.
At design time, the question can be raised, can we make our programming greener?
I started programming in the late 80's, where I've learned to program to avoid memory usage (ok that lead us into the 2000 problem...), and optimize the program to run as smooth as possible with the scarce resources in mind.
With the ever increasing speed of processors, we are spoiled and tend to program sloppy nowadays, who cares about extra loop cycles, just add an extra processor.
Actually when we look nowadays at the 'size' of an application, the term SLOC (Source Lines of Code) is used. In short, instead of measuring the effectiveness of an application, we only look at the size, so size really matters....
Let's go back to the drawing table start thinking about programming again, how can we make our applications more efficient in the usage of resources?

I think green management of an application should be part of the design and run phase, and should already be added in the offerings we bring to our clients. Cars have an energy level, so why don't we green label our applications?

The Quest for the Crane

Once in a while there something which really makes you tick.
Six months ago I went out to do some running, it went bad, so at the end I was in a bad mood. When I wanted to drive back, the road was blocked by birdwatchers, but even though I like bird watching, I was annoyed by them and didn't care about the bird they were looking at. Back home I checked a bird watcher site and realised that it was about a Crane (grus grus), a very rare bird in The Netherlands. I went back but then, of course the crane was gone. Slowly I started getting the crane fever. I started studying web material and books. The biggest chance of spotting one is in the migrating season, from October to December, but when you study the statistics, Cranes are spotted in peaks, hundreds to thousands in one day and zero the day after, and mostly in the south and east of The Netherlands, not my region...
So I kept an eye on the spotting sites, and went out hiking, but apart from a nice walk, no result.
The last week however a Crane was spotted in the Rosmalen region, so I took a 'sebatical afternoon' and went out looking. And fortunately luck was with me, so the Quest for the Crane was completed. In the picture above it is the beautifull purple bird in the middle.

The only thing now is that I don't think the fever now is over...