Software program and programming

    Software programs have been there for many years now, and have advanced a lot in terms of technology, programming, designs and practices.  There are a large amount of population who are interested in learning programming and some find it difficult.  I hope programming related posts in my blog are simple to understand and are helpful in their journey of learning.

“A software program is a set of coded instructions that can be run on a computer to perform the specified operations.”

Fun software program

    Here the computer is any intelligent hardware device that reads the provided instructions and performs the requested operation.  For example, it can be a desktop computer, a laptop, a mobile phone, a tablet, an iot device, or an embedded computer processor, or a virtual hardware in the cloud, etc.

    As the technology is advancing everyday, many of the devices that we use everyday are becoming smart by embedding a computer and a software program.  For example smart TV, smart refrigerators, smart cars etc.

    The operations performed by a software program could be a compute operation on a given data, or to manage data, or to route the data over network, or to control the hardware and resources etc.  A software program is written to solve a user/business requirement and the instructions coded in the software program are aligned to perform operations as required for a given user/business needs.

    For example, an ecommerce software program has the set of instructions having logic to process the submitted shopping order request.  It takes the user order data, computes the total price of the products purchased, applies discounts, processes the payment and generates order confirmation.

  There are too many software programs in this world. There are software programs for providing ecommerce services, for generating financial report of a company, or for schools to manage the students academic records, or for game enthusiasts, or for connecting your mobile phone with your television, or for providing infrastructure service on the cloud, etc, etc, etc.

“A software program is called as a software product, if it has well defined user interface, documentation and support on how to use and workflows.”

    The software product is also interchangeably called as software or software application or simply an app.  Based on its use, different terms are used such as system software, application software, enterprise software etc.  Few examples are, Google Pay, Microsoft Word, Angry Birds game, Amazon Alexa, etc.

    More than one software that have similar user interface and integrates with each other are packaged into one bundle called Software Suite. Software Suite is generally designed in such a way that its collection of software provides a holistic experience to the user with different services offered together.  Few examples are Microsoft Office 365, Adobe Creative Cloud, VMware vCloud suite, Apple macOS etc.

    In my opinion a software program is more of a technical term, and I would use the term software program mostly in the technical documentation.

“The process of writing a sequence of coded instructions to prepare a software program is called as software programming or simply programming or coding.”

    To write a software program, you need to understand the user requirement, define the program design and translate the requirements into sequence of instructions with some logic.  You will then test written software program to verify whether the results are satisfactory, and this process is called as software testing.  There are a lot of programming practices and styles followed in the industry.

“The instructions in a software program are written in a formal language called programming language.”

    There are many programming languages with various levels of abstraction.

    A software program that will be run on a computer should have instructions in the language that the computer understands.  This is called machine language or machine code and these are binary codes. It is impractical for anyone to write big software programs in machine language.

    The assembly language is symbolic representation of machine code instructions that is human readable.  An assembler software translates the assembly language program into the machine code format that the computer understands.  These are low level languages and are very much tied to instruction set of the computer.  For example, 8051, 8086, x86, x64 assembly languages.  It is still impractical to write big software programs in low level languages. 

    To simplify writing a software program, the high level languages were developed.  The high level languages provide higher level of abstraction from machine language and uses natural language elements to simplify programming.  The software program written in high level language is translated to machine language using compiler software programs. You would then run the compiled software programs on the computer.  There are many high level languages and each have different features and styles for writing a software program.  For example, C, C++, Java, Python, C#, etc.

Software programs in different languages

    The software program that is written in any of above language is called as source code and the compiled code that runs on the computer is called as binary executable or simply binary.  The binary executable is then packaged and released to users along with documentation.  The software development methods and practices is a very vast topic to be included here.

Disclaimer:  I hope this information helps someone.  Above writing reflects my view based on my experience and learnings.  Please comment if you find any discrepancy.

Thread dump in Tomcat

In a multithreaded environment if the implementation is not perfect, then deadlocks are common scenarios. We can analyse the deadlocks by placing log statement at proper locations in the code and analyzing the log, however to exactly pinpoint a deadlock thread dump would be required. Using the thread dump, we can exactly point out what are the two threads which are fighting for the monitor and prints the stack trace of each thread. With the stack trace we can easily find the use case where the deadlock occurs. There are many ways of getting the thread dump and few of them are listed below.

Unix:
Thread dump can be obtained in the appropriate log by sending the command
kill -3 PID, where PID is the process id of the Tomcat jvm.

Windows:
If your tomcat is running as a windows console, then you can open the console and type <ctrl>+<break>. This would print out the thread dump on the console. Make sure you increase the screen buffer memory size to capture the entire thread dump. However If your tomcat is running as a windows service, then you can obtain the thread dump by using the Tomcat monitor.

More on Tomcat as Windows service:
The Tomcat service monitor when started will bring an icon on the system tray. Right click on the icon and select "Thread Dump" in the menu options. This will trigger the generation of thread dump and the thread dump stack trace will be logged to the file system based on the Tomcat service configuration.

With the default settings, the tomcat log redirection would be set to auto. This can be verified by opening the Logging tab of the Tomcat service editor. With the default configuration, the threaddump would be either in jakarta_service, stdout or stderr logs. Or if you have configured tomcat service to redirect to a specific file then you will find the stack trace in the configured file. However if you not configured the logging properly for the tomcat service, then the stack trace would be lost. You will have to configure the service with proper values for logging and restart the tomcat service.

There are also other hacks of sending the <ctrl>+<break> signal to the tomcat service. For instance there is an application called SendSignal.exe which can send <ctrl>+<break> signal to tomcat service by running the command: sendsignal PID, where PID is the process id of the tomcat service. The thread dump stack trace would be logged to file system based the tomcat service configuration. However it may not work in all scenarios based on the security settings. Atleast I couldn't get it working as it always gave me the error related to insufficient privileges, though both the service and sendsignal were run by the same user.

NOTE: sc queryex <service_name> will give you the process id of the service.

Tomcat as a Windows service

Tomcat, a very familiar name in J2ee world. An excellent example of how open source is growing so powerful in the server domain.

In Windows, Tomcat can be setup in two ways

1. Install the Tomcat as a standalone server

2. Install the Tomcat as a windows service

The first method is fairly simple, which is nothing but just extracting the tomcat archive and running the startup script to run the Tomcat. Normally in a developers local system he would just extract the archive and run the Tomcat, however for running the Tomcat on the server the second method is preferred. Again there are two ways of installing Tomcat as a Windows service.

1. If you are running the Tomcat installable exe file then follow the instructions to install the Tomcat as a service.

2. If you have just extracted the Tomcat and wish to setup Tomcat as a Windows service or If you have already installed the Tomcat with the default name and would like to install another Tomcat instance with different Windows service name, then there are set of steps one should follow. Below are more details of the same. Lets take a look of it one by one.

Under the Tomcat bin directory you would find two interesting executable files. Assuming that you are using Tomcat with 6.x version, however if you are using different version then the file names would be different accordingly.

Tomcat6.exe - This is a service application for running Tomcat6 as Windows service

Tomcat6w.exe - GUI application for monitoring and configuring Tomcat services.

By invoking these executables with specific parameters we can install, update, edit, run, stop, monitor, delete the Windows service. However as these parameters are more techincal, Tomcat also provides a batch script, service.bat, with two simple commands install/remove. The usage of this batch script is as below.

Usage: service install/remove [service_name]

The service_name is the name of the Windows service and is an optional parameter. If you do not specify the service name then by default it will attempt to install/uninstall the service with name tomcat6.

If you are planning to write a batch script of your own, similar to service.bat, then refer to Windows service HOW-TO in locally installed tomcat documentation or in the internet with link http://tomcat.apache.org/tomcat-6.0-doc/windows-service-howto.html. It would also help you by looking at what all are being configured in the service.bat script provided by Tomcat.

Tomcat6.exe is used to install, edit, run, stop and delete the Tomcat service. However if you are looking for a GUI interface for editing the service configuration then the answer to this would be Tomcat6w.exe. Tomcat6w.exe is used only for configuring and monitoring the Tomcat services, so it is a prerequisite to have the Tomcat service already installed for running Tomcat6w.exe. There are two modes of running this application.

1. Run as a GUI based service editor

2. Run as a GUI based service monitor

By just running this exe without any parameter it would by default attempt to open the service editor with service name as file name without 'w'. Here it would be Tomcat6, assuming that you are running Tomcat6w.exe. If your service name is different, i.e., other than the Tomcat6, then you can open this service configuration by running the command: Tomcat6w //ES//<service_name>. Or much simpler solution is to make a copy of this executable file and rename it as <service_name>w.exe. In this case by just running this new file it would open the editor for windows service with the name as <service_name>.

To run the Tomcat as a monitor, you need to specify the command as: Tomcat6w //MS//<service_name>. It would be beneficial if you write a simple batch script, say monitor.bat, under Tomcat bin directory with just above command for monitoring purpose. Again remember if you have made a copy of Tomcat6w with service name as said above then the command can also be <service_name>w //MS//

In summary, below steps help you in managing the Tomcat as a windows service.

1. Install: In command prompt, cd to Tomcat bin and run the command: service install <service_name>

2. Edit: Make a copy of file tomcat6w.exe and rename it to <service_name>w.exe. Just run this file to open service editor.

3. Monitor: Prepare a batch script with command: tomcat6w //MS//<service_name> (or) <service_name>w //MS//. Run this script to open service monitor.

4. Uninstall: In command prompt, cd to Tomcat bin and run the command: service remove <service_name>

Note: The Windows service name and service display name are different and service name is case insensitive.

SoapUI for Data Setup - Use Case

With Soap UI having very useful features, I find this tool helpful for more and more purpose than just testing and coverage. In one of the product implementation, we were trying to setup the initial necessary data (basic and test data) in easy and best way. We needed the initial setup of few organization profiles and users to proceed further in the product implementation. We had three options - 1. Create manually through UI (painful), 2. Create through web service, 3. Write stored procedures (complicated). As second option seemed easy and clean we built the web service requests and ran each of them to setup data. However we felt there was still some more scope for improvement on this. And then flashed the test suite functionality of SoapUI!!

Once I had the idea it was very easy to setup the project on SoapUI with one test case under a test suite. That's it. I created one test step (web service request) for each profile to be created. However user creation required some of the profile information. Property Transfer came in handy to transfer the required values from the response of a specific profile creation request to the user creation request. (I tried many ways to do this with Property Expansion, however couldn't find a way to do this).


The snapshot of the setup is as shown in the figure. As you can see there are multiple SOAP requests, one each for profile/user creation. The property transfer worked fine, however I observed one weird behavior. Though the response xml did not contain any namespace prefix, In Property Transfer I still had to declare the namespace and mention the xpath elements with this namespace prefix. I still need to find a reason for this behavior.

Another notable thing to mention here is to turn on the option to close http connection after every request. Normally SoapUI will open a http connection and sends the web service requests one by one. However due to few unknown problems on data size limit I encountered, the test case was failing at some point midway through the test steps. However after turning on the option to close http connection after every request the data setup ran very smoothly. This option can be accessed under File -> Preferences -> Http Settings.

SoapUI - Property Expansion

While profiling one of my web services I was looking for an effective tool for load testing my web service. Initially thought of JMeter, but due to my happy experience with SoapUI in the past I decided to use SoapUI.

My request xml had multiple fields however couple of fields I had to send the same data. This wouldn't have been a problem during unit testing, but while load testing this service the value of these fields should be unique per test request. So my problem was say If I am sending 50 requests then in each request the value of these fields though common should be unique. Example request would be as below...

<soapenv:Envelope xmlns:soapenv = "http://schemas.xmlsoap.org/soap/envelope/" xmlns:web = "http://www.xyz.com/webservices/">
<soapenv:Header/>
<soapenv:Body>
   <web:MigrateOrdersRequest>
       <web:Header>...</web:Header>
       <web:Credentials>...<web:Credentials>
       <web:Order>
           ...
           <web:OrderNumber>PO10001</web:OrderNumber>
           ...
           <web:OrderLines>
               <web:OrderLine>
                   ...
                   <web:SerialNumber>PO10001</web:SerialNumber>
                   ...
               </web:orderLine>
               <web:OrderLine>...</web:OrderLine>
           </web:OrderLines>
       </web:Order>
   <web:MigrateOrdersRequest>
</soapenv:Body>
</soapenv:Envelope>

Here I have to pass both PO number and Serial Number unique and same. I initially looked for groovy script, but didn't work out for me. The other feature which really impressed me was Property Expansion which really did it. The property expansion will evaluate the property for you in the soap request may be it is from implicit objects, properties or any random math expression.

The solution initially looked tough was very very simple in the end. I just created a new TestCase in my soapUI testSuite with just one test step which was the soap request. And I created a load test step in the same test case (visibly load test will have only one test step). I used simple strategy with limit as 5 seconds and Test Delay of 5000 milliseconds and giving as many threads as required to mimic the scenario where 'n' (say 50) test requests hit the server. With other strategies where Test Delay was lesser than limit, the counter was not actually unique as the same thread after Test Delay was creating a new request. The modified request is as below.

<soapenv:Envelope xmlns:soapenv = "http://schemas.xmlsoap.org/soap/envelope/" xmlns:web = "http://www.xyz.com/webservices/">
<soapenv:Header/>
<soapenv:Body>
   <web:MigrateOrdersRequest>
       <web:Header>...</web:Header>
       <web:Credentials>...<web:Credentials>
       <web:Order>
           ...
           <web:OrderNumber>PO1000${=context.getProperty("ThreadIndex")}${=context.getProperty("RunCount")}</web:OrderNumber>
           ...
           <web:OrderLines>
               <web:OrderLine>
                   ...
                   <web:SerialNumber>PO1000${=context.getProperty("ThreadIndex")}${=context.getProperty("RunCount")}</web:SerialNumber>
                   ...
               </web:orderLine>
               <web:OrderLine>...</web:OrderLine>
           </web:OrderLines>
       </web:Order>
   <web:MigrateOrdersRequest>
</soapenv:Body>
</soapenv:Envelope>

Here after every load test the PO1000 needs to be modified to give next set of unique numbers. The catch here is that ${=<expression>}was actually evaluating to a value by soapUI at runtime. This expression can be any expression with the objects available in the context like properties, implicits etc.

Database design - basics

Working

Design - Singleton Pattern

JVM is a virtual machine where you find threads, objects, classes, classloaders, garbage collector etc having their own living space in the machine. Of all these it is the threads which keeps the metabolism going. When all the threads are dead the virtual machine dies. During its lifetime these threads create, edit or act upon multiple objects. Objects hold data and defines the behavior. In certain instances we need specialized objects holding data which needs to be shared across the virtual machine. This requires us to maintain only one instance of the object which is shared across multiple threads in the jvm. This is defined by Singleton pattern. Below are my thoughts and understandings of this pattern.

The basic definition of this pattern is as below.
1. Make the object constructor private. This allows the control on object creation.
2. Define the static object reference to self to hold the instance of this object.
3. Define the static getter method to get a reference to the object.
There are many flavors of pattern implementation, which are as below.

A. A very simple way of implementing Singleton is as below. This works in both the situations of single and multi-threaded environment.

public class Singleton{
   private static final Singleton instance = new Singleton();
   //Other data attributes go here...
   private Singleton(){}
   public static Singleton getInstance(){
      return instance;
   }
   //Other methods go here...
}

The above method is simple and very effective, but this may not be desirable in few design situations. The above method does early initialization of the object while the below methods talk about the lazy initializations.

B. When more control is needed, like handling the situations when the initialization in the constructor throwing exception. Though exception may be handled in the constructor or a new exception can be thrown from constructor, we may want to return null value to the to the calling method rather than return this partially initialized instance.

//CASE: Single threaded environment:
public class Singleton{
   private static Singleton instance = null;
   //Other data attributes go here...
   private Singleton(){}
   public static Singleton getInstance(){
      if(instance = null) instance = new Singleton();
      return instance;
   }
   //Other methods go here...
}
//CASE: Multi threaded environment
public class Singleton{
   private static Singleton instance = null;
   //Other data attributes go here...
   private Singleton(){}
   private static synchronized Singleton createInstance(){
      if(instance == null){
         //Exception handling can be done here.
         instance = new Singleton();
      }
      return instance;
   }
   public static Singleton getInstance(){
      Singleton singleton = instance;
      if(singleton = null){
         singleton = createInstance();
      }
      return singleton;
   }
   //Other methods go here...
}

C. There is also the other reason why we may not do the early initialization. If we wish to extend from the Singleton class. Below style talks of this in the multi threaded environment.

//CASE: Parent:
public class Singleton{
   protected static Singleton instance = null;
   //Other data attributes go here...
   protected Singleton(){}
   private static synchronized Singleton createInstance(){
      if(instance == null){
         instance = new Singleton();
      }
      return instance;
   }
   public static Singleton getInstance(){
      Singleton singleton = instance;
      if(singleton = null){
         singleton = createInstance();
      }
      return singleton;
   }
   //Other methods go here...
}
//CASE: Child
public class ChildSingleton extends Singleton{
   //Other data attributes go here...
   protected ChildSingleton(){}
   private static synchronized Singleton createInstance(){
      //Overriding the createInstance behaviour
      if(instance == null){
         instance = new ChildSingleton();
      }
      return instance;
   }
   //Other methods go here...
}

However be careful in the above method as the protected access specifier indicates that it can be accessed by the other classes in the same package as well, which gives a leak to create multiple instances.

D.One of the good approaches I have come across. This works well in all situations and is lazy initialization as well.

public class Singleton{
   //Other data attributes go here...
   private Singleton(){}

   public static Singleton getInstance(){
      return SingletonHolder.INSTANCE;
   }
   //Other methods go here...

   //Inner class
   private static class SingletonHolder{
      private static final Singleton INSTANCE = new Singleton();
   }
}

But beware that Singleton may not really be a single instance and there can be many leaks. There are some more good articles on this topic. Few of them are mentioned below.
http://en.wikipedia.org/wiki/Singleton_pattern
http://www.javaworld.com/javaworld/jw-04-2003/jw-0425-designpatterns.html