In a recent blog post we explored Apache OFBiz Performance Benchmarking. Since then we have executed Apache OFBiz 13.07 load testing. Today’s tech update looks at how performance benchmarking can support the load testing process, and reviews both our testing processes and test results.
Q. Why is OFBiz 13.07 load testing important?
Load testing is an effective means of assessing the performance of an E-commerce/ERP solution. Key purposes of OFBiz 13.07 load testing include:
- To to benchmark Apache OFBiz 13.07 against industry performance standards
- To help the user evaluate the suitability of the system configuration and the logical architecture
- To help the user evaluate the overall robustness of the codes/system
- To check the threshold capacity of the E-commerce system
- To determine what sort of load increase will push the hardware configuration to its limits
- To help the user estimate the minimal hardware configuration that should be able to manage the same load
Q. Which OFBiz 13.07 load testing process did HotWax utilize?
While the process outlined below could be used to test nearly any E-Commerce /ERP solution, our OFBiz 13.07 Load Testing process involved: 50,000 visits for a typical E-Commerce business doing 1,000 orders a day at 2% conversion rate.
Q. Which performance processes were tested?
In performance testing, it is important to identify processes that are useful to test and which are key to the E-commerce cycle. (For our OFBiz 13.07 load testing, we followed OFBiz business processes and use case artifacts as outlined on the official Apache website for OFBiz Wiki.) Today’s tech update includes a detailed analysis of these use cases along with the JMeter test plans and instructions. Additional details are available here:
- E-commerce (Includes basic features like Product Search, Order by Registered User and Anonymous User)
- Order management (Orders by Order Manager)
- Warehouse management (Includes Quick Ship, Pick, Pack and Receive Inventory)
Q.Which parameters were selected?
The OFBiz 13.07 load testing parameters we used (listed below) are the same ones noted in a previous performance benchmarking blog. We included key parameters that can affect system performance and that that mirrored the actual production system usage:
- Registered customers
- Anonymous customers
- Administrative users
- User concurrency
- Key user transactions and their frequency
- User growth
- Peak and off-peak time periods
- Normal business hours
- Conversion rate (percentage of customers completing order process)
Q. What were the user count totals?
Total user counts in each use case for running 24 hrs test plan were as follows:
|Use Case Name||User counts|
|Product search||50000 visits|
|Order by anonymous user|
Order by registered user
Order by phone channel
|Packing and shipping process||1|
|Bulk receive inventory||1|
Note: The exact number of fulfilled orders would vary depending on the picklist and availability. For our test purposes, a single user executes the pick-pack process and orders are filled one-by-one.
Q. What technical architecture was used?
For our OFBiz 13.07 load testing process, we used logical architecture for a single machine multiple user configuration with splitting async and sync jobs over two OFBiz instances (as outlined in previous tutorial blog posts). Our load test simulated a server running an E-commerce store receiving 1,000 OPD with a 2% conversion rate. We executed Apache OFBiz 13.07 load testing within both a hypothetical and an actual technical environment in regard to the following parameters:
- CPU Idle: A computer processor is described as idle when it is not being used by any program.
- CPU Usage: CPU usage can be monitored to see how much of the processor’s capacity is in use.
- Disk I/O: Disk I/O encompasses the input/output operations on a physical disk. The total number of individual disk IO requests completed over a period of one minute
- Bits I/O: The total number of bits sent to the disk (write) and retrieved from the disk (read) over a period of one second
- Load Average: The system load is a measure of the amount of computational work that a computer system performs. The load average represents the average system load over a period of time. It conventionally appears in the form of three numbers which represent the system load during the last one-, five-, and fifteen-minute periods.
Did Apache OFBiz 13.07 measure up?
The short answer is yes. After executing OFBiz 13.07 load testing as described above, our assessment is that it is a robust and full-featured E-commerce/ERP system with excellent end-to-end functionality. For detailed results, please refer to the charts and graphs shown below.
Detailed Load Testing Results
Below are the Hyperic reports detailing the execution of the use cases for 24 hours:
The CPU usage graph shows that it is using a minimum of 20% and a maximum of 32% of CPU capacity during the test. This indicates that the performance of the server with a 1,000 OPD load is satisfactory. The server can facilitate an additional load during the peak season or peak time.
Here the CPU usage seems less, so it would be logical to wonder why we did not use a machine with a lower configuration such as 1CPU. The reason behind this was to keep the value of all other key performance key parameters in within standard range.
Load Average 5 Minutes
The value of Load Average lies between 0.25 and 0.65. This is also a satisfactory result, it should be in between 0.0 – 1.0 range as standard, so a reading below 0.70 means it is working well. In fact, the system is capable of taking an additional load of up to 25% more during peak time.
The above graph shows how much memory was free during the 24-hour period of the test. The memory used was between 1.6G and 2.5G out of total available 3.8G RAM. There is buffer of an additional (approximately) 1G of RAM available during peak time. The results are quite satisfactory here as well.
Ashish began his career as a Software Engineer with Aditisoft Solutions & Consulting Services Pvt. Ltd. (now HotWax Media Pvt. Ltd.) in 2003, and is now Vice President of Operations. Ashish is the local manager of offshore development efforts for HotWax Media. With an emphasis on providing training both to internal HotWax Media employees as well as to clients, Ashish has worked on a variety of projects for clients in North America, Europe and India. He continues to be dedicated to the Apache OFBiz open source project since 2003 and became committer in 2007. He is also a member of Project Management Committee for Apache OFBiz project. Ashish graduated in 2003 from RGPV University, Bhopal (M.P) India, with a Bachelor of Engineering(B.E) degree in Computer Science.