Wednesday, December 21, 2005

Stateless Session beans kill Object Orientation??

Stateless Session beans kill Object Orientation??

Recently I had a chance to have close look at OO principles as I had to prepare to deliver a lecture for a ‘advanced OO development’ course. This allowed me to have a back look on OO after working in several commercial J2EE projects. I started thinking… in those projects, have we used proper OO with real encapsulation, inheritance and polymorphism principles??? Answer was “NOP”…


In most of the J2EE projects people tend to use more service oriented approach rather than going for a object oriented approach due to the J2EE technology limitations. For an example most of the business objects are just plain DTOs where Stateless session beans acts as service providers for those DTO business objects (as an EJB best practice, DTOs are used to transfer entity bean data across tiers :D ). Most of the so called “DTO business objects” are just having a bunch of getters setters encapsulating the attributes. Is this real encapsulation??? No… we need to have the object operations to encapsulate the attributes of our objects. But with DTO business objects we are separating out the object data from its operations causing less encapsulated objects. Making the things worse, stateful session beans did not proved itself in the industry, making OO much away from the J2EE projects.


At least do we get inheritance properly? Session beans do not support direct inheritance making polymorphism also to be impossible. Having no polymorphism in our business objects directly affects the design quality. We end up with lot of if-else clauses handling different variations of our business object types. It is true that there are tips and tricks to get around with EJB inheritance problems, but that is not we need as poor OO programmers. We need tools (J2EE) to help us; we don’t have much time here to help the tools…


So where are we heading??? EJB 3 seems a good improvement but still it doesn’t provide all we need. Some times it seems to be bias at J2EE server vendors but not at the java developers (may be as the spec committee is mostly driven by application server vendors). Anyway I wish to talk more on EJB 3 and OO principles in another post.


Considering all, for me it seems going for Spring Framework is still a better approach, especially when you do not need your system to be distributed across several VMs. In my opinion, stateless session beans still are a superior technology for a locally distributed system (When it is not local there are much better solutions like XML over HTTP). When we think of such locally distributed systems we should think those distributed components as services, not as objects. Internally within components we have to follow clean OO principles. In summery we can develop our logic on plain well object oriented object model and expose the operations (if needed) as services over stateless session bean. I believe that approach gives us a better, and maintainable system.

Wednesday, November 30, 2005

Add Business Intelligence to Your System

Add Business Intelligence to Your System

Recently I involved in designing a reporting system and hence have done fair bit of research on the available reporting mechanisms and frameworks. Although the requirement was to use .NET and Windows technologies, I thought of looking in to some other reporting platforms (in java and open source world) as well.

First thing I got in to my mind was 'Eclipse BIRT framework'. I remembered looking at the BIRT proposal and architecture about a year back. Also I remembered it had proposed a complete reporting life cycle management system. So I visited Eclipse site and saw that they have released the first stable version of the system and it seemed to be very promising. You can access Eclipse BIRT home page here and a very cool flash demo here. (java programmers must see this demo if Microsoft was able to surprisee you with their Visual Studio productivity features :D )

Eclipse BIRT architecture provided me with a very good understanding of the functionality of a report system. Then I thought of having a look at 'Jasper Reports'’. Jasper also seems to be a rich reporting framework and I got a good understanding of reporting internals by looking at report samples and tutorials of Jasper reports.

So after looking in to couple of open source reporting frameworks then I was ready to get in to my real job, which is to explore .NET related reporting technologies. Visual Studio 2005 comes with two major reporting approaches. One approach is to use '‘Crystal Reports'’ version shipped with Visual Studio. This is a limited version compared to '‘Crystal Reports XI' but good enough for most of the reporting requirements. The '‘CrystalReportViewer'’ component provides easy embedding of reports to web forms or win forms. Basically if you have a good '‘Crystal Reports'’ experience and working knowledge on Visual Studio, you will be able to generate quality reports and embed them in to your web or windows application. One great this I noticed was that Crystal Reports engine is able to generate quality HTML reports having cross-browser support even in complex report navigation requirements (which ASP.NET 2.0 is failed to do in some cases).

The other approach is to use '‘Microsoft Visual Studio 2005 Reporting'’ features with or without 'SQL Server Reporting Services'’. In this approach Visual Studio provides a Ă‚‘ReportViewerĂ‚’ component which can be used either in local-mode (no reporting server involved) or in remote-mode (with SQL Server Reporting Serviback-endning as a backend report server).

Local-mode is seems overlapping with the '‘Visual Studio 2005 Crystal Reports'’ and good for simple reporting architectures. Local-mode provides you with more control over the data as you can bind datasets, custom objects in to your reports. If you want more complex/extensible system having report servers running you need to look in to '‘SQL Server Reporting Services'’ and use 'ReportViewer'’ remote-mode. In this mode the data binding and report rendering is done within the report server and the report will be exposed as a web service consumable by '‘ReportViewer'’ components embedded in web forms or win forms. Also SQL Server Reporting services can directly expose html reports with out a separate UI layer in place. Microsoft has done many improvements to their previous '“SQL Server 2000 Reporting Services'’ with their new releases of 'SQL Server 2005'’ and '‘Visual Studio 2005'. Even though these technologies are not very matured as '‘Crystal Reports', they seem to be very powerful and robust with the good industry impression they have on the '‘SQL Server 200 Reporting Services'’.

One thing I noticed after doing this research is that the reporting technologies have become much matured today with all sorts of user/developer friendly tools and complex/extensible architectural-styles. So it seems the true sense of the word '‘Business Intelligence'’ is not so far from today...

Sunday, November 06, 2005

Automate Your Builds and Tests

Automate Your Builds and Tests


This is going to be an ice-breaking post on my blog after a long silence. Couldn't write a post for some time as I was bit busy with a lot of work after coming back to Sri Lanka.

Last few days I was involved in automating build process of a J2EE project in Eurocenter. The project basically has a Struts based web front end, EJB/JDBC based db/business tier and a web service (XML over HTTP) interface for mobile devices. We have come up with an automation plan which will be implemented within next month and it seems progressing well curruntly.

So I thought of sharing some stuffs on the automation implementation of our project as it might help someone else in automating their products. As the first step we have to have a rough plan of our automation process. The plan used in our project goes as follows:

Write a one-step build script: Use Ant or Ant variants to come-up with a build file with will generate the complete build in one command. You may even consider using Ruby build scripts (Rake) for this purpose. (But we should be careful is selecting a immature technology for a core part like the build system of a project.

Add unit-test execution to the script: You should carefully select the unit tests which are properly written to run as stand alone tests. If you have followed my previous postings I have discussed some techniques explaining how to make your code testable. We can have an ant task which will run the unit test cases/suits on the compiled code.

Come up with a push button release: Once we have the build script we can extend it to produce single command releases. Basically this involves checking-out the code from a CVS tag and running the build task. This extended build script can be used by the build manager in producing builds for QA testing as well as producing production builds.

Schedule builds with automation engine: Here we go with automation. First we need to decide on a build engine. In the open source world there are several good automation engines such as ‘Cruise Control’, ‘AntHill’ and ‘BuildBot’. We have decided to go with ‘Cruise Control’ as it seems to be the most popular choice in the java community. In this step we configure our build engine to checkout all the code from CVS periodically (only if any modification is done after the last build) and build the product and run the unit tests. Once we have completed this step we track any build failures (such as compile errors) and unit test failures. In our project we have scheduled builds to run every one hour and this helps capturing introduced bugs within a period of one hour.

Establish reporting mechanisms: Through out the process we should be having basic notification mechanisms like Email and web console for reporting build status and failure notifications. All the build engines support these basic reporting mechanisms and all we have to do is to configure those services to suit the need. For an example if a build failed we can configure ‘Cruise Control’ to send notification emails to the developers who have committed code to CVS after the last successful build.

Automate integration testing: Here we attempt sub component integration in a test environment. In our project we wanted to make sure that the product components are integrating and deployable on the server. The integration testing will ensure that all the components are properly deployable (e.g. EJB testing) and the component communication channels (e.g. JMS queue testing, database connections) are established correctly. Most of the tests in this stage can be considered as in-container testing and test frameworks like ‘Cactus’ are good candidates to support the testing in this stage. In this stage we run “JBoss’ as our application server then deploy our application there and run integration test scripts.

Automate functional/acceptance testing: In this stage we add end-to-end testing to our automation. In our case basically this means performing tests on the web UI and web service interfaces. These test cases are written based on the end-user actions (based on use cases). Example test case is ‘user log-in to the system and changing his password’. In our product ‘HttpUnit’ test cases are written to simulate and validate user actions. Where ever response time is critical we may add performance test scripts written using ‘JMeter’, ‘JUnitPref’, etc… to ensure system response time.

Add code analyzers: Here with each build a code analyzer will inspect the code for “smelly code pieces”. With each build the team lead will get code analyzer report on any added/changed code fragments.

After coming up with an automation plan we can start implementing the plan. Once implemented, we get a set of actions which will run in a sequence in every scheduled build cycle. For you to get better understanding I will summarize the action steps we have in our project build cycle.

1. Checkout sources from CVS
2. Build the application from scratch
3. Execute build verification tests (unit tests)

4. Run code analyzer tool scripts

5. Install a ‘JBoss’ server instance

6. Configure the ‘JBoss’ instance
7. Create the test database

8. Populate test database with test data

9. Deploy the product over the JBoss/file system

10. Startup the server

11. Monitor log files for successful server startup

12. Execute integration test scripts

13. Execute acceptance test scripts

14. Shutdown the server

15. Clean up the build resources

16. Report build/test results


Once we got all the basics in place this is the time to play around extreme automation experiments. For example I have heard some project teams having flashing red bulbs in the development environment if a build failed. Trying these kind of extreme practices can bring team moral up towards testing/automation as well as can take attention of the rest of the project teams (in a good sense
).

Sunday, September 25, 2005

Security with Java Cryptography Architecture

Security with Java Cryptography Architecture

In the last week I was going through some discussions on how to provide security on our new application we are building at Scali AS. In our architecture the server is a C deamon which uses OpenSSL for security where as the client is a Java RCP which uses JSSE.

One observation I had in working with java security frameworks is that Java has lot of Security components, but most of the developers has no very clear understanding of where those components fits in to the application requirement they have. So I thought of having a blog post which introduces the Java security components and their usage.

Here we go... Broadly there are several components belonging to the Java Cryptography Architecture (JCA) (or simply Java Security). JCA can be divided in to several sub categories as follows :

a) Java Authentication and Authorization Services (JAAS)
b) Java Secure Socket Extension (JSSE)
c) Java Cryptography Extension (JCE)
d) General Security Services (Java GSS-API)
e) Java Certification Path API
f) Java Simple Authentication and Secure Layer (Java SASL)

JAAS is the Java solution (framework) for authenticating and authorization uses and services. JAAS provides user-centric as well as code-centric access control mechanisms. JAAS can be considered as a framework and an API focused mainly for non-connection oriented (stand-alone) application security. It is not about encrypting data but it is a implementation neutral API for authentication and authorization (For an example if we decide to move our usernames/passwords from a properties file to a database, still JAAS wil allow us to do this with out changing application code).

JAAS authentication mechanism is pluggable so vendors/users may write there own ‘login modules’ to read/validate authentication credentials from their preferred source (e.g. database, directory service). As I know, JDK does not provide many default ‘login modules’ but generally application servers come-up with several reusable login modules such for reading “prop-files”, “simple database tables”, etc...

JSSE provides a framework and implementation for securing your data in network communications by introducing security later on top of TCP/IP. J2SDK 1.4 onwards supports SSL 3.0 and TCL 1.0 protocols. The most popular SSL model used on internet is to authenticate only the server identity and use encryption on data. Client identity is authenticated using username/password provided by the client (e.g. a typical e-commerce web-site) JSSE provides a convenient API on top of java sockets for enabling your TCP connection security.

JCE consists of a framework defining java security components such as data encriptors, key generators, etc… Sun has provides a default implementation called “SunJCE” for this framework (For example RSA, DES encryption algorithms, DES key generators are provided with the ‘SunJCE’ provider). We may use this default SunJCE (any other third party provider) to provide some cryptographic operation such as encrypting a data stream using “DES symmetric key” encryption.

Java GSS-API allows java developers to make their programs to work with Kerberos based authentication. Similar to JSSE, GSS-API provides authentication, encryption and integrity protection on your data. The underline mechanism used is the widely known Kerberos security model. Kerberos provides strong token based security and supports single sign-on capabilities for the applications. (GSS API mechanism is defined as a Internet Standard under RFC2853)

Java Certification path consists of abstract framework classes which allows user to create, validate, manipulate ‘Certification Chains’. Certification chain is a ordered list of certificates which will establish a trustable mapping from ones identity to his ‘public key’. Java Certification Path API also consists of a default implementation for working with X.509 certificates.

Java SASL is the java implementation of the SASL internet standard defined under RFC2222. As the name suggest the SASL standard also provides authentication and secure transmission of data (so it also can be categorize with GSS and JSSE). SASL is defined to be a light weight protocol and used by LDAP and IMAP implementations. Thus any application intended to work with those applications may need Java SASL API for security integration.

Thus the bottom line is that, we should decide on the technology to be used based on our application context. As a rough example if my application is TCP socket based I may go for JSSE, if my organization security is Kerberos based then I go for GSS-API, if I need to communicate with a LDAP service I may need to consider Java SASL API and so on…

Sunday, September 18, 2005

Ruby blocks, C# delegates, C function pointers and Java inner classes

Ruby blocks, C# delegates, C function pointers and Java inner classes

During the weekend I spent some time to look in to the Ruby 'Blocks'. Blocks are a Ruby language construct which adopts the concept of method references in a very useful manner. Even though this is not a new idea I haven’t seen a language adapting and using method references to the extent as Ruby does. Let’s look at how Ruby adapts this concept. One of the Ruby’s powers is that it takes fewer lines of codes compared to most of the other languages. For an example in Ruby iterating on an array and printing the elements will look like the following:

array.each { |animal| puts animal }

Actually what happens here is, to the "Array.each" method we pass the code block "{ |animal| puts animal }" so “array.each” method will call the passed code block for each element of the array (more information). As u may see this is not exactly passing methods as references. Instead of passing an external method as a parameter, in Ruby we write our code inline as a code block.

Let’s look at some other languages to see the way they allow developers to pass code blocks as parameters of a method.

In C# we may gain the same power with Delegates. Delegate is a type that describes a method signature. Methods with the same signature can be stored in a delegate and can be passed as parameters or be invoked. But delegates are more structured, complex and requires more coding compared to Ruby blocks.

Function pointers of C allows developers to pass functions as parameters to other functions. They are not type safe as C# delegates so the developers are able to pass functions with different signatures as parameters.

Java has no concept called method references and it does that using interfaces and classes. We need to define our code block inside a class and pass that class to the method as a parameter. For an example we use this in implementing listeners, callback routines, and to pass utility code blocks such as compare routines to ‘sorting’ algorithms. Mostly we see these code blocks are implemented as inner classes or anonymous inner classes in java.

If you are a regular reader of my blog you may remember the post “Intermediate Representations vs Software Performance”. There I have explained how to pass a code block to a DAO so that as and when the DAO encounters a database record it will invoke the passed code block. If you look at the way of Ruby’s “array.each” method works you can see both these implementations share the same concept to a some extent.

So I may say ruby has adopted the concept of passing code blocks to a method and made it a really useful construct in the language.

Sunday, September 11, 2005

JMeter in Web Application Testing

JMeter in Web Application Testing

Last week I was discussing with Kartik on performing JMeter performance testing on the Ripple product. They were facing some difficulties in HTTP session handling in their JMeter test plan. This conversation allowed me to refresh my knowledge on this powerful testing framework. So I thought of making a note about JMeter on my blog. This will be a quick and short overview on some of the general JMeter capabilities.

When considering the non functional testing we often hear people talking about different types of tests. Some of those are “Load testing”, “Performance Testing” or “Stress Testing” on the application under test. First of all it is important to know the meaning and differences of these terms used in application testing.

a) Load testing is applying a heavy load on the application and monitors the behavior of it under a heavy load.

b) Performance testing may be carried out to ensure the application responses within acceptable timing on the requests it’s getting.

c) Stress testing is for ensuring fault tolerance capability of the application. For an example we may restart the database while application is in operation and check whether the application can recover under such a condition.

Let's have a quick look at JMeter. JMeter can be readily used in Load and performance testing. The test scripts we may create are called ‘Test Plans’. We can have create several test plans to test our application. These test plans can be used even as automated tests by running JMeter in non graphical mode.

There are several types of elements in a JMeter test plan. JMeter uses ‘Samplers’ to produce requests. It has ‘Logic controllers’ so we can control our test flow with them (e.g. to repeat a request). ‘Listeners’ are used to capture test results. Delays may be introduced by its ‘Timer’ elements.

Session handling is an important activity in any practical test plan. For general cookie based sessions we may use a Cookie manager to keep our cookie based session information across the requests. But when URL-Rewriting is used in session management we have to use “HTTP URL-Rewriting Modifier’ to keep session information across requests.

For me one of the most interesting elements in JMeter is its assertions. For an example returning HTTP 200 by our server doesn’t always mean that the request has been served successfully. That may be a user friendly error message returned by the server when a critical error occurred. We need to be more specific and probably wants to check the returned html to validate the server response. This is made possible with ‘Assertions’ in JMeter.

When configuring our test plan sometimes we face difficulties in simulating the exact browser behavior with JMeter. One handy tool that would help here is ‘TCPMON’ tool comes with Axis utilities. We can capture and compare http communications of JMeter with the communications of browser easily with this tool.

It is really interesting seeing JMeter as a part of a automated testing environment together with JUnit and HTTPUnit test frameworks…

Tuesday, September 06, 2005

Use of Visitors in Producing Data Streams

Use of Visitors in Producing Data Streams

Few days ago I saw a cool implementation of WBXML parser by Uchitha combining Visitor pattern and Composite pattern. This blog is a note on how we can use Visitor Pattern in producing output streams in a flexible manner.

We will take a specific example of data manipulation to demonstrate the idea. Assume a client and server communicating via a TCP socket. Server expects a specific protocol from the clients communicating with it. In our example assume clients can request two operations from the server.

a) read files from the server
b) execute commands on server

Further assume, before doing any of the above operations clients need to login to the server, and after the operation client logout from the server. We use Java TCP socket to send/receive data from/to server.

Say for an example we need to read a file from the server. Assume steps specified in the protocol are as follows:

1) send to server “LOGIN [username]
2) read from server’s “WELCOME
3) send to server “READ /tmp/myfile.txt
4) read from server “FILE-LENGTH=245
5) read from server next 245 bytes (which is the actual file data)
6) send to server “THANKS
7) read from server “BYE

All these commands are ASCII encoded and send/receive through a TCP socket created with the server. If we look at above conversation
step 1, 2 are related to ‘login’,
step 3, 4, 5 are related to ‘file read’ and
step 6,7 are related to ‘logout’ protocols.

By separating those steps in to separate entities we allow reuse of entities as well as we may combine them to perform more complex communication patterns. Ok… So far so good… but where is the visitor pattern?

Basic idea here is we have a visitor (Java TCP Socket) visiting a set of entities (Login, Read Files, Execute and Logout protocol handlers) to perform a certain task. All the protocol handlers have implemented a simple interface called “IVisitableEntity”. “accept” method is ready to welcome a visitor which will be the socket instance in this case.

public interface IVisitableEntity {
public Object accept(Socket visitor);
}

Concrete classes of this interface will be LoginProtocol, FileReadProtocol, ExecProtocol and LogoutProtocol. Here is an example implementation of LoginProtocol. This will handle the steps 1 and 2 specified above.


public class LoginProtocol implements IVisitableEntity {

private String username;

public LoginProtocol(String username) {
this.username = username;
}

public Object accept(Socket visitor) {
//get out put stream
OutputStream os = visitor.getOutputStream();

//get input stream as a BufferedReader so we can read full line once
BufferedReader in = new BufferedReader(visitor.getInputStream());

//write the login request to the socket
os.write(("LOGIN " + this.username).getBytes());

//read the response and validate it
String response = in.readLine();
validateResponse(response);

//we do not return any thing in this entity
return null;
}
}

In the same manner we can implement FileReadProtocol, ExecProtocol and LogoutProtocol concrete classes which will handle their own protocols atomically.

Now lets see how we assemble complex operations with these atomic protocols we have… say we have to implement a process which needs to read a file on the server and then executes a command on the server. This simple process may be coded as follows.

public void process() {
Socket socket = getTCPSocket();//obtain the socket to server

//login to the server, username as a parameter
LoginProtocol loginProtocol = new LoginProtocol("hasith");
loginProtocol.accept(socket);

//read the remote file content
FileReadProtocol fileReadProtocol = new FileReadProtocol("/tmp/myfile.txt");
byte[] fileData = fileReadProtocol.accept(socket);

//call the command on remote server
ExecProtocol execProtocol = new ExecProtocol ("command -o param");
execProtocol.accept(socket);
}

Look at the above implementation. Our socket act as a visitor, it visits different entities and those entities will accept the visitor and perform read/write on the socket according to the protocol.

Here are some notes on the pattern:
a) As all the protocol entities are implemented the “IVisitableEntity” interface we can even define the process flow dynamically at runtime.

b) Logic (protocol) is separated from data structure (socket data) fulfilling primary goal of Visitor pattern.

c) Logic entities are atomic and reusable in making complex process flows.

d) Also we may wrap the visiting socket to provide more extensibility. For an example to make the socket access thread safe.

This pattern may be also used on converting xml structure in to a plain text stream as I mentioned in the beginning of the blog. There, due to repetitive nature of xml elements and attributes we can also combine “Composite Pattern” to provide more power to our converter.

Saturday, August 27, 2005

Distributed Computing and SOA

Distributed Computing and Service Oriented Architectures

Few days before I had a discussion with Jostein from Markus Data Norway regarding distributed computing architectures. These days he is involving in designing the system architecture for next generation Procasso system. We were discussing about inter-component and B2B communication technologies such as RMI, Web Services, Remoting, ASPX and their usage in different scenarios. So I thought of making a blog post on Destributed computation and discuss to where it is heading today.

We know OOP approach has made a major revolution in programming models. It has been great success and therefore people tend to look at every problem in an object oriented way. So people came up with the concept of distrubuted objects to solve the distributed computing (DC) problem. Then we started to hear about technologies like java RMI, DCOM, CORBA, .NET Remoting used in solving destributed computing problems.

But in practice organizations started facing lots problems with OO approach of DC, and the messaging systems such as IBM MQSeries, MS MQ technologies, Web Services became much more popular due to the flexibility it provides.

So what is the problem with Destributed Objects. In my openion the concept it self has some problems. Think in this way.. The interface my organization provides to the outside world should present a service but not row data. For an example If my organization is specialized on creditcard processing then I should provide a creditcard processing service to the outside world. But objects are the representation of my business entities and they should be manipulated withing the organization boundry to provide a service. That is on the theory side.. Now on to practical issues..

Distributed Objects (DO) provide very convinient way for developers to work with as they can think and work on remote objects in a similar way as they work with general objects. But the problem is tight coupling between communicating parties. When we talk about tight coupling there can be different type of coupling in DO approach.

a) Code Coupling : In Distributed Objects approach both parties share common interface, i.e actual code. We see DOs works well in the lab, but once u deploy it on production, it is very defficult to evolve one side independent of the other side due to sharing of code. Assume i want to provide a new method on my DO interface because my new business partner wants a new service from me. Addition of this new method will break all my contracts with the existing partners as I have to change my interface code.

b) Time Coupling : DO works in a synchronous manner, i.e the caller waits for the callee to finish the operation. This is obviously not the way to perform a B2B operations.

c) Location Coupling : Assume u call a business method of another company but the response you need to direct to another party not back to you. This kind of addressing issues are not possible to handle with DO model.

On the other hand Web services being XML (text) based messaging technology provides a loosely-coupled communication betweens two parties. We do not share code but messaging meta-data (WSDL). We can define our transport mechanism making asynchronous invications possible. We see Web Services running on HTTP, SMTP, JMS and even on plain TCP. Web service addressing specification provides convinient way to loosen the location coupling between parties.

Due to these reasons we see Service Oriented Architectures based on loosely coupled messaging systems taking over the interconnection domain. But still I think DOs will still has a major role to play on inter-process or inter-component communications withing a sofrware system.

Saturday, August 20, 2005

Sunday, August 14, 2005

Make your design testable - Test driven design

Make your design testable - Test driven design


Everyday we hear great deal of words like "test driven development", "unit tests", "test-first programming" etc.. I know every developer loves if they can cover 100% of their code with unit tests. But most of the time, the complain is that they do not have enough time for writing unit tests.

In my view the root cause to the problem is not actually the time, but the design of our components. Most of the developers cant find enough time to write tests for their components, just because it is almost impossible to write test cases with the component design. Most of the time, the way our component is integrated with its environment makes it impossible to simulate the dependencies on a test environment.

Next obvious question is "How do we write our components to be testing friendly". One good answer to the question is "Use Dependency Injection". Two of the most popular injection mechanisms are "Constructor Injection" and "Setter Injection". Unlike their big names the concepts are pretty simple :) . Let me take an example.. Let's take a situation which is generally considered fairly difficult to write test cases, i.s testing UI workflows....

In our example the business flow will be as follows;
-- fetch all customers from the database via a DAO
-- show customer list to the user on a GUI View
-- user will pickup a customer from the list
-- send a billing mail to the customer using MailSender service


Our simple workflow will be dependent on a DAO, a GUI View and aMail sender service for it's operations. The business workflow under test will look like the following:


class BusinessProcess {

private ICustomerDAO dao;
private IView view;
private IMailSender
mailSender;
........
.....
public void perform() {

Customer[] customers = dao.getAllCustomers(); //get all customers from db

view.setInput(customers); //ask ui view to display customerlist

view.open(); //this method is blocked until user selects a customer

Customer selectedCustomer = view.getSelected(); //get user selected customer

mailSender.sendBillingMail(selectedCustomer); //send the billing mail


}

}



One important think to notice is inside the business process method we should not perform any dependency lookups. It will just use the instance variables dao, view, mailSender representing dependent services. These instance variables are initialized either by Constructor injection, or Setter Injection. In case of constructor injection, the constructor of the "BusinessProcess " will look like the following


class BusinessProcess{

private ICustomerDAO dao;
private IView view;
private IMailSender
mailSender;

public BusinessProcess( ICustomerDAO dao, IView view, IMailSender mailSender) {

this.dao = dao;
this.view = view;
this.mailSender = mailSender;

}

}





Next important point is that our BusinessProcess is not dependent on any Implementation of the services. It just dependent on the interfaces ICustomerDAO, IView and IMailSender. This will allow you to write mock services and inject those mock services to the business process inside your test environment.

If you take the above design it is easy to write a test which will simulate complete business process even including user interactions. U will need to write mock objects for ICustomerDAO, IView and IMailSender services and inject those to the BusinessProcess at your test case.

Probably by now you got the basics of a testable design. Once you get your design correct then it will be a fun writing tests to your component. Always when you design think about the testability. It will not only make your components testable but will improve the extensibility of your component. Also this will be a good starting point for a test-first development practice :) ....

Thursday, August 11, 2005

Intermediate Representations vs Software Performance

Intermediate Representations vs Software Performance

During last few days some of my friends at Scali-Norway were working on writing a Data Access Objects (DAOs) to access system data. They were writing their DAOs in C++ and used a third party driver to access the postgress database. In the simplest scenario DAO reads rows in a table and converts them to a intermediate object representation and returns the collection of objects. Problem they are facing now is the less performance and high memory usage. By going through their code I found that they would have avoided these problems if they have designed the DAOs correctly at the first place. Anyway in this post I will discuss how to represent your data in an intermediate representation without compromizing performance or memory usage.

To demonstrate the concept I will take a simple example. Assume u have a "customer_data" table. U need to send mails to all the customers who have not paid their bills for the last month. U need to interact with an external billing service to check whether a particular customer has paid or not. Also for extensibility u need to wrap customer's data in a "Customer" object which is your intermediate representation. Think a while about a design that you will propose to solve this problem....

The approach my friends have taken is as follows:

They have a DAO class which will iterate thruogh the database recordset and create a collection of Customer objects which will be returned to the upper layer.

public Customer[] getAllCustomers() {

Customer[] customers = new Customer[];
RecordSet rs = getSQLRecordSet();
while (rs.hasnext()) {

customers[i] = convertRecordToCustomer(rs.next());

}
return customers;

}


Then in the business method they iterate through the given Customer collection and send a mail if that particular customer has not paid his bill.

public void informCustomers() {

Customer[] customers = DAO.getAllCustomers();
for each customer {

if (customer has not paid the bill)
{
sendMailToCustomer();
}

}

}


If we analyze the above solution, we can see to perform a given operation we need to go through two loops. Say they had 1000 records in the resultset, then first they will be running a loop of 1000 to convert their recordset rows in to intermediate representation. And another 1000 loop to perform actual business process on those records. The worst part is that you load all your data in to a collection of objects and pass the collection to the upper layer. Imagine you have 10M records in your database, you will probably run out of memory in trying to perform the above operation.


But the good news is you can easily find a better solution to the above problem by applying a simple design pattern. The important point here is actually the business process doesnt need all the customer objects at once to be in the memory, instead it needs one at a time to process. So in our new implementation we write our business logic in a listner class implementing a interface called "CustomerListner" as follows.


interface CustomerListner {

public void onRecord(Customer cust);
}

class BusinessProcessAction implements CustomerListner {

public void onRecord(Customer cust) {

if (customer not paid the bill)
{
sendMailToCustomer();
}

}


}

Now we call the business process in the following manner. We create an instance of BusinessProcessAction which will deal with one customer instance and process the logic on it. Then we ask the DAO to notify the BusinessProcessAction instance as and when it reads a data record from the database. Code will look like as
follows..

BusinessProcessAction action = new BusinessProcessAction ();
DAO.process(action);

Our DAO will act as a producer of customer instances. Once we called the DAO.process() method it will start fetching records from the database, convert that record to a Customer instance and then it will ask our BusinessProcessAction to perform the business logic on that customer.

class DAO {

public void process(CustomerListner listner) {

RecordSet rs = getSQLRecordSet();


while (rs.hasnext()) {

Customer customer = convertRecordToCustomer(rs.next());
listner.onRecord(customer);

}

}


}


This will do the job for you. If you closely look at this implementation u can see we have achived all our objectives.
1) Altogether we runs only 1000 loops for 1000 records
2) We haven't load more than one record to the memory at once
3) Most important is.. our business method still uses Intermediate Representation and it is independent of database formats..


Specially when the record production operation is asynchronous this pattern gives a definite edge. Assume in above example fetching data from the datasource takes 1 hour per record.. Then with earlier implementation it will take 1000hour before transfering the control to the business layer. But with the second approach the processing is real-time, as and when a record is fetched from the database a mail will be sent to that customer.


This example is just to demonstrate the concept of the pattern. You can extend this pattern to achive much more flexibility if u think more given a specific problem. If you start looking at problems relating them to this pattern you will be able to save most of the processing time and the memory usage of your programs....

Wednesday, August 10, 2005

Extensibility layers of a "SW Product"

Extensibility layers of a "SW Product"

Few days ago my friend Senthoor and I were discussing on use of Facades for sessionbeans to hide the complexity of EJB 2.x. That discussion made me to think more on the architecture of a "Product" based software system. Here is a little thought on SW product design..

Broadly we can catogorize software projects in to two catogories.
1) Product development
2) Custom service software develoment

In my openion, compared to "Service software architecture", we need to pay special attension on some of the design aspects when designning a "Product architecture". During the continuous development stage of a Product based software,
a) Developers need to customize the product for different customers.
b) Product need to withstand against technological changes which happens over the time.
So by paying a little more attension on following factors in the design phase, we will be able avoid many extensibility problems in the long run.

Layered aspect of a "Product" architecture needs to be designed to support high level of extensibility. In a product, we should distinguish and identify the Kernal layer, customization layer (business process layer), service interfaces, and the presentation layer of our product for better software management.

Kernal consists of fine-grained operations and will include "data access layer" and "utility services" ( In a J2EE project entitybeans and utility sessionbeans performing atomic operations will make your kernal of the product). For an example "sendSMS(to, messge)", "insertStudent(student)" can be two operations in the kernal components. These opeartion should be atomic and should not compose business processes.

It is really important to understant the difference between fine-grained operation and a business process. for an example "insertStudent(student)" is a fine-grained operation where as "registerStudent(studentData)" will be a business process. Your customization layer should compose your business processes. In the above example, "registerStudent" business process may look like the following.

registerStudent() {
insertStudent();
sendSmsToStudent();
chargeStudentBankAccount();
}

You can define your transactions, security constraints on your business process. Also this is the layer which may be customized for different customers of your Product (For an example a customer may request to send a mail to the registered student instead of a SMS). Ideally kernal should not be touched in customizations and should be managed as a seperate sub-project. Kernal can be associated to customizations as binary libraries. This will also force developers to document the components and will improove project documentations :). Also it is not recommend to build the kernal with your customizations in a single build script. But Kernal should also be evolved by adding new features which are required by new customizations in a seperate sub-project.

Another important thing is to hide the technology used in your business layer from the presentation layer. This can be achived through simple facedes. For an example you may write POJO model Facades to hide your sessionbeans and let developer to instanciate Facade with "new" operator and call business methods. Your facede will hide JNDI lookups and EJB specific exceptions, etc.. This will allow the developers to consentrate on business logics rather than configuring framework specific stuffs. On the other hand this allows you to switch the technology managing your business components with out impacting your presentation layer. (e.g. switching from EJB2.x to EJB3)

In this blog posting I tried to discuss some design concepts related to the layered view of a "Product" software architecture. In my openion, even in a "Service Software" architecture, extensibility should be provided to a certain extent , but there you may have to compromize it with the cost and performance.....



Monday, August 08, 2005

Creating a remote file system browser, SWT vs Swing

Creating a remote file system browser, SWT vs Swing

Few days ago some of my friends at Scali AS, Norway were wondering of how to write a file select dialog in java, to browse a file system of a remote machine. They were writing a RCP application based on Eclipse framework and working with SWT/JFACE widgets. Remote machine was a Linux server which acts as the front node to a Linux Cluster.

Given the problem my first thought was to extend and override the SWT FileDialog class or related content provider classes with my own implementation that fetchs the remote file structure over SSH. So I went throgh the SWT source codes to find out the feasibility of this solution. As many of us know that the SWT uses higher level OS dependent graphical components and those are rendered by the operating system. Therefore I didn't find any way to solve my problem with SWT libraries. "Open" method of the SWT org.eclipse.swt.widgets.FileDialog just calls the native OS method. So I was not able to plug my own file system model to the SWT FileDialog.

So obviously the next choise was the Swing JFileChoser component. As the file browse dialog needs to be just a popup, it is possible to call Swing FileChooser withing a SWT view classes by a general java call.

During the analysis I found that the Swing classes are well designed based on the MVC model. Swing components are designed to be crosspatform so the model hides the platform differences from the view. In the case of Swing file select dialog, the View is implemented as "javax.swing.JFileChooser" class and the "javax.swing.filechooser.FileSystemView" class provides the file system model to be used. Swing has different FileSystemView sub classes for different OS'. (examples WindowsFileSystemView and UnixFileSystemView).

So it seemed to be ideal for my requirement. All I need to do is to provide a my own subclass of FileSystemView(model) and pass an instance to JFileChooser(View) constructor. My subclass of FileSystemView will fetch remote file structure through SSH and render it according to the FileSystemView interface agreement.

Further analysis showed that the FileSystemView(model) interface uses java.io.File instances heavily to model file objects. In my requirement the File objects will not represent real file instances of the hosting machine but logical view of a file on the remote machine. For this reason I wanted File objects to behave in a disconnected manner from the hosting OS.

But unfortunately some methods like "exists()" of the java.io.File class is bound to the OS via security constraints. The problem is these methods of java.io.File class are badly designed to repersent only the real file entities of the hosting OS. It is not designed to work in a disconnected nature. If they had moved the OS dependent operations on to a util class and made File to be used in a disconnected manner then problems like mine would be solved perfectly. (Here someone may argue that is not the OOP way, but still it will make java file framework much more extensible)

Anyway even with some constraints it is possible to implement my idea with Swing JFileChooser where SWT fails to do so. This simply shows the extensibility provided by Swing compared to SWT. But SWT itself has many pros with it. In my expierience SWT provides a better user expierience in the sence of porformance and look n feel. Also the APIs are more natural for simple GUI programming.

Thus in my openion these two technoligies can be (and should be) used to complement each-other and not to substitute each-other....

Sunday, August 07, 2005

J2EE seems to be on the right track with EJB3 architecture

J2EE seems to be on the right track with EJB3 architecture

J2EE at its early stages was designed to cater a world having a component market place. The extent to which it has materialized this objectve is doubtful. Anyway it is clear most of the time J2EE is used to solve a complex business problem rather than to write course-grained components that can be plugged seemlessly between systems.

If we look at the EJB2.o or EJB2.1 frameworks they seems to be designed to favour big J2EE vendors not the poor J2EE developers :) . The complexity of the platform required complex tools in supporting the development. Further more developers were spending most of the time solving complex J2EE configuration issues rather than solving the business problem in hand.

Few of the most significant drawbacks of the EJB2.x architecture are
- its non-ability to test the business components out side the container.
- dependency lookup nature of JNDI requred high amount of coding in refering a resource.
- entity beans are not serializable so DTOs are to be used causing more coding and less performance.
- deployment discriptors are hugely complex and has a steep learning curve.
- making the business component classes completely independent of the EJB framework classes is not possible even with a complex patterns due to framework abstractions and wiered checked exceptions of the framework. (For an example you could write a POJI as a framework independent interface for your business object and let EJBObject interface to extend it, if the RemoteException class was a unhcecked exception)
- EJB-QL was unable to solve most of the complex querying requirements
... and more... and more....

Some times I was thinking like Sun may not have any people actually doing projects in J2EE, even they involve designing J2EE archtecture. The EJB2.x design is that much cumbersome and developer frustration was that high specially when your company doesnt want to spend on complex tools for J2EE development.

Any way as the time passes, Some open-source products like Spring for business tier and Hibernate for persistance seemed to be right tools compared to EJB2.x. Assembling your own container with Tomcat, Spring and Hibernate was seemed to be the better way for data centric web projects. Both hibernate and Spring uses POJO model, making out side the container testing to be very mush easier (to be more precise they dont run withing a context of a container).

Spring uses Dependency injection pattern to inject any resources required by the components. The setter injection model has prooved to be very useful specially in unit testing. Here you can find a very simple example demonstating the concept behind Sping framework. Also the light weightness of Spring and Hibernate attracted most of the frustrated EJB2.x developers.

But today EJB3.0 is well designed to solve most of the EJB drawbacks it had in early versions. It has learnt very much from Spring and Hibernate. Persistance model is completely redesigned following the Hibernate model. In EJB3 you enterprice beans are POJOs and not dependent of any framework classes. It largly make use Annotations and avoid use of abstractions as much as possible.

In my openion this trend brings a hope to a much better form of frameworks. if I remember right a framework is defined a set of abstractions on which developers can extend. Hibernate/Spring uses reflection to avoid framework specific abstract chasses and intefaces... EJB3.0 uses annotations and avoid use of framework classes as much as possible. This allows the developers to consentrate more on the actual business logic.

In future blogs I will describe how EJB3.0 integrates most of the powerful features of Hibernate and Spring to make our lives easier. For me it seems with EJB3.0 Java communty is well armed to face M$ challenge ...