Recently, we have discussed Load Testing Apache OFBiz with Apache JMeter and Technical Design for Apache OFBiz load testing with Apache JMeter. In this blog post we will look into some of the aspects of performance benchmarking for Apache OFBiz based enterprise software solutions.
Performance benchmarking has grown into an important step in software development process, especially in this era of web based software, mobile computing, and cloud computing. With enterprise scale software solutions like Apache OFBiz, performance benchmarking becomes even more critical. Apache OFBiz has a large set of business processes, which when deployed result in numerous client/server and database interactions. If business reporting is enabled this load further increases. With such a large number of parallel transactions running at one time, it is important that the application’s architecture is robust and when deployed it performs smoothly. Performance benchmarking is a similar process to software development, in that you need to have a good understanding of the business domain and its requirements to be able to perform it. Below we will discuss some of the aspects of performance benchmarking in the context of Apache OFBiz.
Business Use Case Analysis
Apache OFBiz has a large set of business processes, which processes will be utilized depends on your business requirements. Further, Apache OFBiz could be used in a variety of ways and you might be looking for a customized business solution on top of Apache OFBiz. It is important to clearly lay out business use cases before starting your performance tests. Knowing all of the business use cases, makes it easier to analyze and draw you performance expectations out of your business. Apache OFBiz is a popular solution for implementing e-commerce ERP systems, below are some sample use cases for a standard e-commerce ERP system.
|#||Process Name||Description||Process flow|
|1||Product Search||User comes on the site and searches for a few products. In this way, the user visits multiple pages of the site in one visit and stays for 2-5 minutes. In this process the user stays on each page for 30-60 seconds.||1) Home page
2) Keyword search
3) Product detail page
|2||Order by Anonymous user||User searches products on site and places an order with multiple items as an anonymous user.
User follows the steps of product search/selection and then continues to checkout.
|1) Add products to cart
2) View cart
3) Fill in the details on the checkout page
4) Review order
5) Submit order
|3||Order by Registered user||User searches products on site and places an order with multiple items as a registered user.
User logs into site and follows the steps of product search/selection and then continues to checkout.
|1) Log into the application
2) Add products to the cart
3) View cart
4) Fill in the details on the checkout page
5) Review order
6) Submit order
|4||Quick Ship Approved Orders||Order manager selects approved an order and quick ship entire order to fulfill the order.||1) Log into the order manager
2) Find the approved order
3) Select the order and go to the order view page
4) Fulfill the order using Quick Ship option
|5||Order by Order Manager||Order manager creates an order on behalf of customer from the back-end.||1) Log into the order manager
2) Select a customer to place the order
3) Add products to the order
4) Select payment and shipping options using the Quick Finalize option
5) Review order
6) Create order
|6||Receive Inventory||The Facility manager receives product inventory.||1) Log into facility manager
2) Select a product to receive
3) Receive the desired quantity
Business Usage Analysis
This is an important step in performance benchmarking. This is where you define your performance expectations from your ERP system. It is important that the parameters derived in this step are close to the actual production system usage.
As we have seen above, Apache OFBiz can be implemented and deployed in unique ways as per your business needs. Once you know your business requirements and use cases , it becomes easier to understand how your system will be used, what the key business processes of your system are, the types of users and their numbers, workload on various business processes and subsystems, etc. With the introduction of front end e-commerce, the usage definition changes altogether. There could more hits on your e-commerce portal with many concurrent, active sessions. The response time of key functions like category navigation, product search, and checkout becomes more critical. Below are some key parameters that could help you analyze the usage of your e-commerce ERP system.
- Registered Customers
- Anonymous Customers
- Administrative Users
- User Concurrency
- Key User Transactions and Their Frequency
- User Growth
- Peak and Off Peak Time Periods
- Normal Business Hours
- Conversion Rate (Percentage of Customers Who Completes Their Order)
Once you have your business requirements laid out, you can determine system requirements such as availability, capacity, response times, serviceability, etc. These serve as a basis for designing your logical deployment architecture.
The use cases and usage parameters also serve as the basis for designing your test suites and test scripts. Once your technical architecture and test scripts are in place you are ready to deploy your performance tests and analyze the results.
Performance analysis is an iterative process which not only helps you extract the best performance out of your system by removing any blockers and glitches, but it also helps you benchmark your Apache OFBiz solution against industry specific performance standards or your own set of custom standards.
Below are some of the key parameters that help you analyze the performance of your Apache OFBiz e-commerce ERP System.
- Hits per Second: Hits Per Second shows the number of concurrent users on an Apache OFBiz e-commerce ERP Subsystem. Technically the number of HTTP requests triggered per second to perform a certain business activity.
- Sessions per Hour: This metric helps in the analysis of capacity requirements. The system should be able to handle the number of unique visitors that are about to hit the store in an hour.
- Requests Served per Second: Requests served per second are the number of HTTP requests fulfilled in a second. It could be that a server has completed the requested activities and has now acknowledged with the HTML page.
- Page-Views per Second: Pages viewed per second are the full pages accessed by the user (i.e., requests, not including embedded objects such as images and CSS files). This parameter can be defined when an existing reference system is available and OFBiz can match the existing system in terms of workflow.
- Number of Active Sessions: Active sessions can be best explained as the time spent by a user over the web page. It includes asdditional conditions like a user may be in a waiting state, a request submitted to the system, or reading a web page. This may be determined based on business requirements or through analysis of web server logs and analytic data for existing reference systems.
- CPU Usage: This metric helps in analyzing the hardware performance referencing the system hardware (processor) and how much work it is doing.
- Load Average: The load averages differs from CPU percentage in two significant ways; load averages measure the trend in CPU utilization not only an instantaneous snapshot, as does percentage but it also includes the overall demand for the CPU, not just how much was active at the time of measurement.
- Memory Usage: You could analyze the memory consumption over time for the server. This could help in troubleshooting performance bottlenecks and optimization.
- Bandwidth Usage: This is another check to determine network utilization. Network plays a significant role in thew performance of web applications.
To conclude, for seamless performance of an enterprise software solution, performance benchmarking is a necessity. The process to accomplish it is very similar to the software development process itself, and potentially requires the same amount of effort. Though, if you have the right performance testing framework, along with a standard process to execute it, you are sure to achieve the expected performance from your Apache OFBiz enterprise solution.
At HotWax Media we have a complete framework outlined and the processes defined for performance benchmarking of Apache OFBiz. We are continuously producing new benchmarks, in our future blogs we will keep sharing the results as we make progress. And in the meantime if you are looking for a fully performance optimized and scalable ERP solution built on Apache OFBiz, you have landed in the right spot!