What is a framework?
A framework is a real or conceptual structure intended to serve as a support or guide for the building of something that expands the structure into something useful. Think about a scenario where someone want to construct a house, what are things need to plan before that,
Select a location for your house
Select bank for home loan
Design your home house
Acquire permits for construction
Find the builder
Search for Material cost etc etc....
What is Test automation framework?
Similarly test automation framework is set of guidelines, assumptions, concepts and tools that provide support for automation software testing. The main advantage of framework that low maintenance cost. Any change in test cases due to functionality change or addition will not change the driver script.
Types of Automation Framework
Following are the popular automation framework:
1) Test script modularity framework
It is simplest of all frameworks. The test script modularity framework requires the creation of small, independent scripts that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests, realizing a particular test case.
For example consider logging into Flight Reservation Application and checking whether the application has loaded on successful log-on. Here , the tester will simply record the steps and add validation steps.
SystemUtil.Run "flight4a.exe","","","open"
Dialog("Login").WinEdit("Agent Name:").Set "Guru99"
Dialog("Login").WinEdit("Password:").Set "Mercury"
Dialog("Login").WinButton("OK").Click
//Check Flight Reservation Window has loaded after successful log-on
Window("Flight Reservation").Check CheckPoint("Flight Reservation")
Advantages:
> This is quick way to generate the test script and automation expertise is not required
> You can easily learn the product
Disadvantages:
> Scripts are not reusable and test data are hard coded into the test scripts.
> It is involve lots of maintenance cost
The test library architecture framework is very similar to the test script modularity framework and offers the same advantages, but it divides the application-under-test into procedures and functions instead of scripts. This framework requires the creation of library files (SQABasic libraries, APIs, DLLs, and such) that represent modules, sections, and functions of the application-under-test. These library files are then called directly from the test case script.
Using the same example as above, the function for logging in to Flight Reservation will look like .
Function Login()
SystemUtil.Run "flight4a.exe","","","open"
Dialog("Login").WinEdit("Agent Name:").Set "Guru99"
Dialog("Login").WinEdit("Password:").Set "Mercury"
Dialog("Login").WinButton("OK").Click
End Function
Now, you will call this function in the main script as follows
//Driver Script
Call Login() (This will invoke login functionality)
---------------------------
Other Function calls / Test Steps.
---------------------------
Advantages:
> It will be easy to maintain because high level code can be re-used in structured scripting.
Disadvantages:
> Technical expertise is necessary to write scripts using Test Library Framework.
> It will more time to develop
> Test Data is hard coded within the scripts
3) Data-Driven Testing Framework
Data-driven testing is a framework where test input and output values are read from data files (datapools, ODBC sources, cvs files, Excel files, DAO objects, ADO objects, and such) and are loaded into variables in captured or manually coded scripts. In this framework, variables are used for both input values and output verification values. Navigation through the program, reading of the data files, and logging of test status and information are all coded in the test script.
For example two steps are required to develop “Flight Reservation Login” script using this approach
Step 1: Create a Test - Data file which could be a text file, excel file , csv file or any database source
Step 2: Develop Test Script and access the data from test data file
SystemUtil.Run "flight4a.exe","","","open"
Dialog("Login").WinEdit("User:").Set DataTable("User", dtGlobalSheet)
Dialog("Login").WinEdit("Password:").Set DataTable("Password", dtGlobalSheet)
Dialog("Login").WinButton("OK").Click
//Check Flight Reservation Window has loaded
Window("Flight Reservation").Check CheckPoint("Flight Reservation")
**Note “dtGlobalSheet” is the default excel sheet provided by QTP.
Advantages:
> Any modification in test script will not affect the test data
> Test cases can be executed with multiple data set so number of scenario can be created by changing the input data file
> Easy to maintain
Disadvantages:
> Expertise and more time is required to develop the test scripts and test data.
4) The Keyword-Driven or Table-Driven Framework
Keyword-driven testing and table-driven testing are interchangeable terms that refer to an application-independent automation framework. This framework requires the development of data tables and keywords, independent of the test automation tool used to execute them and the test script code that "drives" the application-under-test and the data. Keyword-driven tests look very similar to manual test cases. In a keyword-driven test, the functionality of the application-under-test is documented in a table as well as in step-by-step instructions for each test.
This framework have 3 basis components:
Keyword (Action) - Keyword is an action that can be performed on a GUI Component. For “Text Box” (GUI component), some action (keyword) would be “Input Text”, “Verify value” etc.
Application Map (Object Repository) - An Application Map Provides Named References for GUI Components. Application Maps are nothing but “Object Repository’
Component Function - that actively manipulate or interrogate GUI component. For example “A function would be click on web button with all error handling” or “Enter data in a Web Edit with all error handling”. Component functions could be application dependent or independent.
Two steps are required to develop the scripts in this framework
Step 1 – Creating the data table. This will contain the GUI components name, Action and arguments if requires
Step 2: Write the script in form of component function
Once you've created your data table(s), you simply write a program or a set of scripts that reads in each step, executes the step based on the keyword contained the Action field, performs error checking, and logs any relevant information. This program or set of scripts would look similar to the pseudo code below:
Function main()
{
Call ConnectTable(Name of the Table)
{ //Calling Function for connecting to the table.
while (Call TableParser() != -1) //Calling function for Parsing and extracting values from the table.
{
Pass values to appropriate COMPONENT functions. Like Set(Object Name , Argument) ex. Set( Agent Name , Guru99).
}
}
Call CloseConnection() //Function for Closing connection after all the operation has been performed.
} //End of main
The advantage of Keyword Driven Framework is that the Keywords are re-usable. To understand this consider you want to verify login operation for a Website say GMAIL. The table will look like this
If you observe in this case the Keywords Set , Click , Verify remain the same for which corresponding component functions are already developed. All you need to do is change the Application Mapping (Object Repository) from earlier Flight Reservation to GMAIL , with change in argument values and the same script will work!
Advantages:
> High re-usability
> Test tool independent
> Easy to maintain
Disadvantages:
> Initial investment will be high but it will realized when application is pretty big.
> High automation expertise is required.
5) Hybrid Test Automation Framework
The most commonly implemented framework is a combination of all of the above techniques, pulling from their strengths and trying to mitigate their weaknesses. This hybrid test automation framework is what most frameworks evolve into over time and multiple projects.
A framework is a real or conceptual structure intended to serve as a support or guide for the building of something that expands the structure into something useful. Think about a scenario where someone want to construct a house, what are things need to plan before that,
Select a location for your house
Select bank for home loan
Design your home house
Acquire permits for construction
Find the builder
Search for Material cost etc etc....
After that only the house construction
is started. Do you think house construction can not be started
WITHOUT these guidelines?? Yes certainly we can but if the guidelines
are followed it will result in beneficial outcome.
Based on above, a Framework can be
defined as a set of guidelines which when followed produce beneficial
results.
What is Test automation framework?
Similarly test automation framework is set of guidelines, assumptions, concepts and tools that provide support for automation software testing. The main advantage of framework that low maintenance cost. Any change in test cases due to functionality change or addition will not change the driver script.
Types of Automation Framework
Following are the popular automation framework:
1) Test script modularity framework
It is simplest of all frameworks. The test script modularity framework requires the creation of small, independent scripts that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests, realizing a particular test case.
For example consider logging into Flight Reservation Application and checking whether the application has loaded on successful log-on. Here , the tester will simply record the steps and add validation steps.
SystemUtil.Run "flight4a.exe","","","open"
Dialog("Login").WinEdit("Agent Name:").Set "Guru99"
Dialog("Login").WinEdit("Password:").Set "Mercury"
Dialog("Login").WinButton("OK").Click
//Check Flight Reservation Window has loaded after successful log-on
Window("Flight Reservation").Check CheckPoint("Flight Reservation")
Advantages:
> This is quick way to generate the test script and automation expertise is not required
> You can easily learn the product
Disadvantages:
> Scripts are not reusable and test data are hard coded into the test scripts.
> It is involve lots of maintenance cost
- Test Library Architecture Framework
The test library architecture framework is very similar to the test script modularity framework and offers the same advantages, but it divides the application-under-test into procedures and functions instead of scripts. This framework requires the creation of library files (SQABasic libraries, APIs, DLLs, and such) that represent modules, sections, and functions of the application-under-test. These library files are then called directly from the test case script.
Using the same example as above, the function for logging in to Flight Reservation will look like .
Function Login()
SystemUtil.Run "flight4a.exe","","","open"
Dialog("Login").WinEdit("Agent Name:").Set "Guru99"
Dialog("Login").WinEdit("Password:").Set "Mercury"
Dialog("Login").WinButton("OK").Click
End Function
Now, you will call this function in the main script as follows
//Driver Script
Call Login() (This will invoke login functionality)
---------------------------
Other Function calls / Test Steps.
---------------------------
Advantages:
> It will be easy to maintain because high level code can be re-used in structured scripting.
Disadvantages:
> Technical expertise is necessary to write scripts using Test Library Framework.
> It will more time to develop
> Test Data is hard coded within the scripts
3) Data-Driven Testing Framework
Data-driven testing is a framework where test input and output values are read from data files (datapools, ODBC sources, cvs files, Excel files, DAO objects, ADO objects, and such) and are loaded into variables in captured or manually coded scripts. In this framework, variables are used for both input values and output verification values. Navigation through the program, reading of the data files, and logging of test status and information are all coded in the test script.
For example two steps are required to develop “Flight Reservation Login” script using this approach
Step 1: Create a Test - Data file which could be a text file, excel file , csv file or any database source
| User | Password |
| Anil | anil |
| Sunil | sunil |
| Tinu | tinu |
Step 2: Develop Test Script and access the data from test data file
SystemUtil.Run "flight4a.exe","","","open"
Dialog("Login").WinEdit("User:").Set DataTable("User", dtGlobalSheet)
Dialog("Login").WinEdit("Password:").Set DataTable("Password", dtGlobalSheet)
Dialog("Login").WinButton("OK").Click
//Check Flight Reservation Window has loaded
Window("Flight Reservation").Check CheckPoint("Flight Reservation")
**Note “dtGlobalSheet” is the default excel sheet provided by QTP.
Advantages:
> Any modification in test script will not affect the test data
> Test cases can be executed with multiple data set so number of scenario can be created by changing the input data file
> Easy to maintain
Disadvantages:
> Expertise and more time is required to develop the test scripts and test data.
4) The Keyword-Driven or Table-Driven Framework
Keyword-driven testing and table-driven testing are interchangeable terms that refer to an application-independent automation framework. This framework requires the development of data tables and keywords, independent of the test automation tool used to execute them and the test script code that "drives" the application-under-test and the data. Keyword-driven tests look very similar to manual test cases. In a keyword-driven test, the functionality of the application-under-test is documented in a table as well as in step-by-step instructions for each test.
This framework have 3 basis components:
Keyword (Action) - Keyword is an action that can be performed on a GUI Component. For “Text Box” (GUI component), some action (keyword) would be “Input Text”, “Verify value” etc.
Application Map (Object Repository) - An Application Map Provides Named References for GUI Components. Application Maps are nothing but “Object Repository’
Component Function - that actively manipulate or interrogate GUI component. For example “A function would be click on web button with all error handling” or “Enter data in a Web Edit with all error handling”. Component functions could be application dependent or independent.
Two steps are required to develop the scripts in this framework
Step 1 – Creating the data table. This will contain the GUI components name, Action and arguments if requires
| Object (Application Map) | Keyword (Action) | Argument |
| WinEdit(User) | Set | Anil |
| WinEdit(Password) | Set | anil |
| WinButton(OK) | Click | |
| Window(Flight Reservation) | Verify | Exists |
Step 2: Write the script in form of component function
Once you've created your data table(s), you simply write a program or a set of scripts that reads in each step, executes the step based on the keyword contained the Action field, performs error checking, and logs any relevant information. This program or set of scripts would look similar to the pseudo code below:
Function main()
{
Call ConnectTable(Name of the Table)
{ //Calling Function for connecting to the table.
while (Call TableParser() != -1) //Calling function for Parsing and extracting values from the table.
{
Pass values to appropriate COMPONENT functions. Like Set(Object Name , Argument) ex. Set( Agent Name , Guru99).
}
}
Call CloseConnection() //Function for Closing connection after all the operation has been performed.
} //End of main
The advantage of Keyword Driven Framework is that the Keywords are re-usable. To understand this consider you want to verify login operation for a Website say GMAIL. The table will look like this
| Object (Application Map) | Keyword (Action) | Argument |
| WinEdit(Username) | Set | abc@gmail.com |
| WinEdit(Password) | Set | hello |
| WinButton(Sign in) | Click | |
| Window(GMAIL) | Verify | Loads |
If you observe in this case the Keywords Set , Click , Verify remain the same for which corresponding component functions are already developed. All you need to do is change the Application Mapping (Object Repository) from earlier Flight Reservation to GMAIL , with change in argument values and the same script will work!
Advantages:
> High re-usability
> Test tool independent
> Easy to maintain
Disadvantages:
> Initial investment will be high but it will realized when application is pretty big.
> High automation expertise is required.
5) Hybrid Test Automation Framework
The most commonly implemented framework is a combination of all of the above techniques, pulling from their strengths and trying to mitigate their weaknesses. This hybrid test automation framework is what most frameworks evolve into over time and multiple projects.
