There are multiple approaches to install JMeter on Mac. You can either do it manually just like we do it in windows or you can follow a more simplistic method to install JMeter via HomeBrew. I prefer the Homebrew method, as it gets the job done quickly with few simple commands. I will especially recommend it to those who are new to Mac OS and don’t want to go in details of manual configurations.

1. Open Mac Terminal, where we will be running all the commands.

2. In terminal, first we will run command to install HomeBrew

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)

At this step, It might prompt you to install mac developer tools first. Just confirm this and provide you mac password if required. Installation process will begin now and once the Homebrew installation is completed It will show success message on mac terminal.

3. Now execute following brew command to install JMeter.

brew install jmeter
or
brew install jmeter --with-plugins

A success message will be shown on mac terminal when the installation is completed.

4. You are ready to use JMeter now. Just type and enter JMeter in terminal to launch it. In case it does not work you can also use following command.

open /usr/local/bin/jmeter

Additional Tips for JMeter Configuration & Optimization:

• Java JRE Installation: JMeter is platform independent application built on Java. Therefore it is important to have correct version of JRE installed in your system for smooth functioning of JMeter. JRE version should not only meet JMeter requirements but also match with your system specs (32/64 bit).
• JMeter Best Practices: JMeter consumes lot of memory, so it is important to follow JMeter best practices. Actual load tests should be executed from command line and listeners should be used intelligently.
• JMeter Memory Tuning: Default JMeter java configuration comes with very low memory allocations. So before running any load tests, first you should update the JVM parameters according to test requirements and your machine’s capacity. These parameters (HEAP="-Xms512m -Xmx512m") can be found in JMeter script file within bin folder of JMeter installation path (/usr/local/Cellar/jmeter/2.xx/bin/). Check this guide for further details on JMeter performance tuning.
• JMeter Mac UI: JMeter provides you option to select JMeter theme that fits well with overall look and feel of Mac OS. You can select it from Menu > Options > Look and Feel > System.

Let me know if this worked out well for you. If you need further help, please feel free to ask in comments section

The post How to Install and Configure JMeter in Mac OS appeared first on Software QA, Functional, Automation & Load Testing with Arif.

1- Setup and Verify Java

Apache JMeter is purely a Java based tool; therefore first of all we need to make sure that Java is properly installed and configured in our system. If you are already using any java based applications, then Java might already be installed on your system. To check if Java is already installed open command prompt and run command java -version.

If Java is successfully installed and configured then the command will return valid java version number (java version “x.x.x.x”)  as shown in image below:

If Java is not installed or configured, then you may get a message like this: “java” is not recognized as an internal or external command, operable program or batch file. 

To fix it, you need to download and install valid java JRE or JDK corresponding to your operating system. Java JRE or JDK can be download from Java Download page on Oracle website. After successful installation of JRE/JDK, open command prompt again and run the command java -version . It should now return java version number that you installed. If you still do not get the valid response after above steps, then you may look into setting java environment variables under windows system properties.

2- Download and Install JMeter

To download JMeter simply go to official Apache JMeter website and download the zip or tgz package.

After downloading the file, just unzip it anywhere in your system. In the unzipped folder there are all the files which JMeter requires for its functioning.

3- Configure JMeter for better performance

There are many ways to optimize JMeter for better performance. It is a complete topic in itself but for starter, I will recommend at least configuring java xms and xmx values in JMeter.bat file. By doing so, you can make sure that JMeter is properly using the available memory resources. It also reduces the chances of OutOfMemory errors.

To change these settings, open jmeter.bat file in notepad and update following lines

// default configuration

set HEAP=-Xms512m -Xmx512m
set NEW=-XX:NewSize=128m -XX:MaxNewSize=128m

// new settings could be like this *depending* on your hardware and software specs
// note that, Max heap size should not exceed the 80% of total system memory

set HEAP=-Xms512m -Xmx2048m
set NEW=-XX:NewSize=128m -XX:MaxNewSize=1024m

Other than this configuration, it is also important to use listeners intelligently in your test plan. Listeners that consume more memory like Table, Tree, Assertion and Graph listeners should be avoided during actual load test execution. It is best to only generate JTL test results file during actual load tests and later that JTL file can be used to create different reports.

4- Launch JMeter & Execute JMeter scripts

There are multiple ways to start JMeter. We can start it in GUI mode by opening jmeter.bat file or we can simply run commands in CMD to execute jmeter scripts without opening it in GUI mode.

4.1) To open JMeter GUI, just navigate to apache-jmeter-2.13>bin directory and double-click ApacheJMeter.jar or jmeter.bat file.

It should open up JMeter in GUI mode as in image below:

4.2) As discussed earlier, JMeter consumes lot of system resources while running in GUI mode. Therefore, it is best to use it in command-line non-GUI mode while performing actual load tests. JMeter scripts can be executed from CMD by using command below:

jmeter -n -t myScript.jmx 

Where...

–n means that JMeter will run in command line mode
-t myScript.jmx mentions the path and Filename of JMeter script

There are other parameter as well which you can pass along with command like path to JTL results file, server IP and port number etc. Command prompt will show progress messages during the script execution and also a summary after successful execution of script. You may also check additional tips for JMeter tuning in my other blog.

Still face any issues while setting up JMeter? Feel free to ask and I will try my best to help out

The post How to Setup and Configure JMeter in Windows appeared first on Software QA, Functional, Automation & Load Testing with Arif.

In this blog, I will do a brief comparison of Top 4 performance & load testing tools, which are considered best by professional performance testers due to their extensive feature set, maturity level and reliable results.

 1. Apache JMeter:

Apache JMeter is an open source performance & load testing tool, which is developed and maintained by Apache. JMeter is mostly used for load testing of web services and web application servers. Testing teams are also known to use it for functional testing of web services. Following are some key points about JMeter.

a) Open-source: JMeter is open source and available free of cost; therefore many software teams prefer to use it for its cost effectiveness.

b) Components: JMeter has test elements like Thread Group, Samplers, Listeners, Pre & Post processors. Other than that, there are tons of third party free/commercial tools which can be integrated with JMeter to enhance its capabilities like JMeter Extras plugin, BlazeMeter, UBIK load pack and Loadosophia etc.

c) Platform compatibility: JMeter is compatible with Windows, Mac and all UNIX based systems.

d) Supported applications: JMeter primarily supports Java & Java server page applications. But it is also equally good to test server applications / web services / databases developed in other technologies. JMeter can also be used for load testing of mobile application server-side.

e) Reporting: In open source/free tools category, JMeter is one of most mature tools as its first was version was released in 1998 and since then there have been many upgrades to it. Thanks to continuous support it delivers reliable results. Reports generated though are very limited and teams tend to use external plugins/tools for more detailed reporting.

f) Available resources: JMeter has one of largest online community which shares helpful information with each other through different forums and software testing blogs. Because of this reason JMeter is considered a great option for beginners in performance and load testing.

2. Micro Focus (HP) LoadRunner:

LoadRunner is a commercial performance testing solution originally developed by HP and now acquired by Micro Focus. It has very advanced set of features which usually do not come built in with open-source or free tools. Some of the key points about LoadRunner are listed below.

a) Commercial tool: LoadRunner is a commercial product, falling in category of most expensive performance testing tools. Micro Focus (previously HP) LoadRunner license cost varies depending on required virtual users, protocols and perpetual licenses etc. Despite its high cost, it is still a preferable choice of enterprise testing teams because of its advanced features and dedicated customer support.

b) Components: LoadRunner is not a single application but a complete suite of tools like VU Generator, Controller, Analyzer, Load generator, Load calculator and protocol advisor.

c) Platform compatibility: LoadRunner can operate from Windows operating system or can be used as a cloud solution.

d) Supported applications: LoadRunner (performance center) provides support for widest range of applications. It can be used to test performance of databases, server side applications and native/browser based mobile applications.

e) Advanced reports: LoadRunner is best known for its very detailed reports which help a lot in analyzing the performance issues.

f) Available resources: Micro Focus provides dedicated support & knowledge base for LoadRunner to all licensed customers. Other than that, one can also find plenty of articles & video tutorials online on Micro Focus LoadRunner.

3. SmartBear LoadUI:

LoadUI is a load testing tool which was introduced by SmartBear after success of famous web service functional testing tool called SoapUI. LoadUI works very well when used with SoapUI for performance testing of APIs and Web services. Following are some key points about LoadUI.

a) Free & Commercial version: LoadUI comes both in free and licensed version. Free version has all the load testing features but the reporting options are limited in it. Because of this option, LoadUI is a good choice for those who want to start free of cost and later want to switch to paid version.

b) Platform compatibility: LoadUI can run from Windows operating system.

c) Supported applications: LoadUI can collect data from servers developed on top of any mainstream technology.

d) User friendly/Interactive features: LoadUI stands out in rest of performance tools because of high usability. It requires comparatively lesser time to understand and operate. Moreover it has features which give a very interactive load testing experience. Test engineer can create, configure and modify tests during the execution. Reports can be examined and analyzed as the scripts/data is changed. This enables tester to understand and trace the real performance issues.

e) Visual reports: LoadUI Pro presents reports in a very easy to understand graphical format. LoadUI free version has very limited reporting features providing only the basic reports.

f) Available resources: LoadUI team has put in a special effort to create video tutorials, blogs, and articles that can be really helpful for learning LoadUI. Sample LoadUI projects are also available on SmartBear website; that can be explored by anyone for practical understanding.

4. IBM Rational Performance Tester:

Rational Performance Tester is a performance testing solution developed by IBM. Rational Performance Tester is a commercial product just like HP LoadRunner. It is often used to test enterprise level applications like SAP, Oracle etc. Following are some major points related to IBM Rational Performance tester.

a) Commercial tool: IBM Rational Performance tester is a commercial product with very high license cost. Because of its expensive price tag it is mostly acquired by enterprises having complex performance testing requirements which cannot be fulfilled by open source or low cost performance tools.

b) Platform compatibility: IBM Rational performance tester can run on Windows, Mac and Linux AIX.

c) Supported applications: IBM RPT supports majority of applications/protocols including Web HTTP, SAP, Oracle, SOA, Citrix, Siebel and TCP. Support for additional protocols can be added by using IBM Rational Performance Tester Extensibility Software Development Kit (SDK).

d) Infrastructure requirements: IBM Rational performance tester has complex infrastructure requirements. Highly skilled and specialized resources are required to setup the test environment.

e) Code-free scripts: Rational performance tester gives ability to create code-free scripts that can be generated by recording test flows. These test recordings can also be viewed in form of rendered HTML.  For advanced and customized actions, java code can be inserted anywhere in the script.

f) Available resources: IBM provides dedicated support to its licensed customers. But other than that availability of help material is quite little, if compared to other famous tools like JMeter & LoadRunner.

End Note:

When it comes to selection of best tool, then there is no straightforward best option that works for everyone. As different tools have different strengths and drawbacks, so it largely depends on factors like project requirements, availability of skilled resources and budget etc. Good amount of time should be spent on selection of right tool as this may prove to be decisive factor for getting the right results.

The post Top 4 Performance Testing Tools Comparison appeared first on Software QA, Functional, Automation & Load Testing with Arif.

To tackle these factors, it is must to have performance testing as an integral part of test plan. We can define Performance testing as “a non-functional software testing technique which determines responsiveness, stability, reliability and resource usage of system under a certain user load”.

Performance testing can be further categorized into different types; which are conducted by testing teams as per project requirements. Before going into details of performance testing types, we will first look into common performance issues found in software applications and the factors that can be used to identify these performance issues.

a)  Load Time: Load time is mostly associated with the time taken by a page to load or the initial time which any desktop, mobile application takes to start completely. This time should be as low as possible. According to industry standards it should be less than 5 seconds in case of web pages and less than 5-10 seconds in case of most mobile and desktop applications.

b) Response Time: Response time usually indicates the performance of an individual transaction or query. It is the time which starts from sending a request till the time of receiving response. Lower response time indicates good performance and higher response time indicates performance issues in application which need to be traced.

c) Performance Bottlenecks: Some common performance bottlenecks are related to CPU, memory and network utilization. To trace and resolve bottlenecks different approaches and tools can be used at different levels.

• at application level, developers can look into code to find poor algorithms and code inefficiencies
• at database level, database administrators can use profilers and improve queries
• at operating system level, system engineers can use performance monitors to find resource usage and ensure that it remains below the industry standard of 60%

Performance Testing Types:

Since the nature of performance issues vary from system to system; therefore we have various performance testing types which are normally performed by professional performance testers to identify performance issues. All these testing types have been given unique names as per their approaches but ultimately all these testing types come under the umbrella of performance testing.

Performance Test: A performance test is any test that can be used to measure performance, capacity, reliability, scalability and/or response/throughput time of software application.

Load Test: It is the type of performance testing which is used to understand performance behavior of any system under a specific user load for specific period of time.

In this approach, load is constantly increased on the system until the threshold value reaches. Increasing load means increasing the number of concurrent users and the transactions they are performing. This approach helps in identifying performance metrics like response time, throughput time, resource utilization etc. Similar load tests can be executed before and after code changes to check the difference in performance metrics. Such usage of load test is also termed as ‘Benchmark performance testing’.

For example, we have a load test for total 1000 concurrent users for total time span of 30 minutes. So, this test can be started from 10 users which can be constantly increased to 1000 concurrent users in time span of 30 minutes.

Stress Test:  It is the type of performance test which is used to measure stability of system by testing it on its upper capacity limits and then pushing it beyond its capacity; while keeping the hardware resources constant. Because of this approach, it is considered a negative type of performance testing; as the objective here is to break the system under the extreme / unexpected conditions.

Software teams use this performance testing approach to identify the system components which fail first under soak test. These components are then tuned and optimized to either do not fail at all or recover more gracefully under extreme conditions.

Spike Test: Spike test is often considered a subset of stress test as in this form of testing; stress on system is put by increasing the number of users suddenly by a very large amount. Spike test can be defined as type of performance test; which is carried out to validate the performance characteristics of system when the load volumes are repeatedly increased beyond expected levels within short periods of time.

The objective of spike test is to determine whether the system will be able to sustain the sudden user load beyond expectation.

Soak Test: Soak test also known as endurance test, is a type of performance test in which load is put on system for a longer duration of time. Soak test duration can range from couple of hours to even a day.

Soak testing is performed because an application/server may work well for an hour or two, but may start giving issues if continuous load is exerted for a longer period of time. These issues normally arise because of memory leaks or because of hardware limitations and soak testing is very useful in finding such issues.

Capacity Test: Capacity test is a common type of performance testing which is used to determine how many users application can handle easily before its performance becomes unstable or unacceptable.

Capacity performance testing is often considered very important as it helps in determining the number of users which the application/server can handle easily. Results from capacity tests can be used to further optimize the usage of hardware resources or tune the application as per requirements.

End Note:

The post Performance Testing: Key Concepts, Issues & Testing Types appeared first on Software QA, Functional, Automation & Load Testing with Arif.