Broken links in ReportNG - Webdriver [FIXED]

ReportNG is a simple HTML reporting plug-in for the TestNGunit-testing framework. It is intended as a replacement for the default TestNG HTML report. The default report is comprehensive but is not so easy to understand at-a-glance. ReportNG provides a simple, colour-coded view of the test results.
This plug-in should be configure in the POM of your project, but I faced some issues with the link created when a test fail. (Log Output).

Researching about the issue I found this possible configuration to solve the issue.

Info:
org.uncommons.reportng.escape-output
Used to turn off escaping for log output in the reports (not recommended). The default is for output to be escaped, since this prevents characters such as '<' and '&' from causing mark-up problems. If escaping is turned off, then log text is included as raw HTML/XML, which allows for the insertion of hyperlinks and other nasty hacks.
Mainly we need to set to false the escape-output property. I know it's not recommended, but it was the solution I found, if any of you have a  better idea, it's welcome.


Issue:
Links are broken in the report, Image locations is fine. 

Solution:

Add this property after <configurations> in your POM file (Maven project):

<systemPropertyVariables>

<org.uncommons.reportng.escape-output>false</org.uncommons.reportng.escape-output>    

</systemPropertyVariables>

After applying this fix:

Clicking on the link, the image will be shown.

Magento error [FIXED]: "Please enable cookies in your browser" just with Internet explorer and Chrome.

There can be some reasons why Magento issues the “enable cookies” page.

In my case, it was because Chrome and IE don't use localhost as a domain for Magento.

The solution was calling my SUT url (testng.xml) using the network IP instead of localhost or 127.0.0.1. i.e 192.168.0.123.

After restarting my isolated environment, the issue was solved.

Conclusion: User your network IP when you are running Magento as localhost for cross browsing testing. Firefox does not face this issue.. who knows why... haha.

Magento... Finally it's running!

Yeah! After some weeks of trying to setup Magento to use it as SUT for some test, I could install it. I used WAMP 2.4 (64 bits) and Magento 1.7. 

Mainly I faced with 2 issues:
1) "php_curl" must be loaded.
2) "database server does not support InnoDB storage engine"

First Issue ""php_curl" must be loaded":

The first scenario was the most complex, because I tried all possible solution that I found in internet but without succeeding.











The first issue was solved removing the ";" manually at this line:

;extension=php_curl.dll

of the files located at:

...\wamp\bin\php\php5.4.12\php.ini
...\wamp\bin\apache\Apache2.4.4\bin\php.ini


But ... I also needed to add these lines to both php.ini files:

extension=php_pdo.dll
extension=php_mcrypt.dll

Then, restart all WAMP services. So.. First issue solved!

Second issue "database server does not support InnoDB storage engine":

This was the fix that should be added to www\magento\app\code\core\Mage\Install\Model\Installer\Db\Mysql4.php, after:

$variables  = $this->_getConnection()
            ->fetchPairs('SHOW VARIABLES');




Apply this fix, then restart all services and lunch again the Magento installation
.

This is the fix:




// Fix
        if (!isset($variables['have_innodb'])) {
                $engines = $this->_getConnection()->fetchPairs('SHOW ENGINES');
                return (isset($engines['InnoDB']) && ($engines['InnoDB'] == 'DEFAULT' || $engines['InnoDB'] == 'YES'));
        }
 // End fix





Now. Enjoy Magento E-commerce.







Hackathon agile Mendoza 2013

Finally Mendoza had its first Hackathon last week in GlobalLogic Mendoza offices.
People sharing their knowledge with just one goal: Make a social application for the World.

After 1 hr debating different ideas we decided to create one application for bus transportation. The name was Busk. 
Initially, the users will be in charge of recording the bus routes once they take the bus, as soon as more users join the project, more routes will be in the system.

BusK is a social application for mobile. This app provides you the bus routes in your town being the users who save them. Some features included in Busk will help to determine which line will take you to your target in less time.
Busk will provide the nearest bus stop to take the bus according to your actual position, in a future, it will calculate the STA (Schedule Time of Arrival) and the time you need to wait for the bus. Once in the bus, Busk will inform you when you should leave the bus to reach your target.

Beers, Food and drinks ready!  ... Go!

We continue developing the app, everyone is welcome to help :)

Info:

F /BusK.transporte 
T @busktransporte
Git: https://github.com/kleer-public/BusK.git

Testers working in an Agile Team goes to Scrum Alliance

The Scrum Alliance is a non-profit professional membership organization created to share the Scrum framework and transform the world of work. I just posted an article about testers in agile teams. Take a look at my article :)

http://scrumalliance.org/articles/541-testers-working-in-an-agile-team

Design Patterns for automation frameworks

Object Page and Factory Page

“Page Factory” is a patter that represents the UI as a Class. Also, the GUI will include some features. This pattern will provide a bridge between the page and the tests.

Here are the main advantages of Page Object Pattern using:

1) Simple tests and ordered

2) Good support to tests saved in one place.

3) Easy way to create new test cases. Test can be created by people without programming skills.

Implementing these patterns in a project: 

Object Page:

Page Factory

New Testers Skills

Roles had changed with the time, now we can find 3 different tester roles:

QA/QC Engineer: They will be in charge of creating and executing manual test. they will validate the external product quality. They will apply functional testing, Exploratory testing, etc.

Test Engineer: They will be charge of scripting test cases using some automation framework. they have some knowledge about programming (initial/intermediate). They could also apply some performance and security testing. 

Engineer in Test: They will be in charge of creating, designing, maintaining Automation frameworks. This role will require some programming skills and they validate the internal quality.

QA Engineer vs Test engineer vs Engineer in test

GTAC 2013 - NYC

GTAC 2013 was definitely more than I expected. In Two days,they included more than 20 awesome presentations.

For my point of view, the best presentation was "Breaking the Matrix - Android Testing at Scale" by Thomas Knych (Google), Stefan Ramsauer (Google) and Valera Zakharov (Google).

Take a look at it and mainly to the parallel execution's tips.
There were many great presentations that we could review later.

Unit test coverage

As a Test Engineer, you should validate the unit tests coverage. Many times, we don't have access to all source code, and  it could be enough to check metrics in a tool like Sonar.
Otherwise, you could read unit tests and determine if programmers are covering the most important cases and suggest new scenarios.

In order to do it, I propose these validations:

Size:
For collections:

• Test with an empty collection
• A collection with 1 item
• The smallest interesting case 
• A collection with several items

Dichotomies:

• Vowels / Non-vowels
• Even / Odd 
• Positive / Negative
• Empty / Full.

Boundaries:

• If the function behaves differently for values near a particular threshold.

Order:

• If the function behaves differently when the values are in different orders. Identify each of those orders.

Example in Python:

import unittest

class TestStockPriceSummary(unittest.TestCase):
    """ Test class for function a1.stock_price_summary. """
 
 def test_empty_list(self):
  """ 
  Return an empty tuple when price_changes is empty.
  """
  price_changes = []
  actual = a1.stock_price_summary(price_changes)
  expected = (0,0)
  self.assertEqual(actual,expected)
 
 def test_single_item_positive(self):
  '''
  Test when the list only includes a positive item
  '''
  price_changes = [2.45]
  actual = a1.stock_price_summary(price_changes)
  expected = (2.45,0)
  self.assertEqual(actual,expected)
 
 def test_single_item_negative(self):
  '''
  Test when the list only includes a negative item
  '''
  price_changes = [-2.45]
  actual = a1.stock_price_summary(price_changes)
  expected = (0,-2.45)
  self.assertEqual(actual,expected)
 
 def test_single_item_zero(self):
  '''
  Test when the list contains only an item = 0
  '''
  price_changes = [0]
  actual = a1.stock_price_summary(price_changes)
  expected = (0,0)
  self.assertEqual(actual,expected)
 
 def test_general_case(self):
  """ 
  Return a 2-item tuple where the first item is the sum of the gains in price_changes and
  the second is the sum of the losses in price_changes.
  """
  price_changes = [0.01, 0.03, -0.02, -0.14, 0, 0, 0.10, -0.01]
  actual = a1.stock_price_summary(price_changes)
  expected = (0.14, -0.17)
  self.assertEqual(actual,expected)
 
if __name__ == '__main__':
    unittest.main(exit=False)

Test case creation, Scripting, Execution, Cross Browsing (Step 5 to 10)

Cross Browsing testing is always a challenge for QA. I'm pretty sure everyone has faced issues with IE trying to run a script created for Mozilla Firefox. 
The following process describes my personal approach to gaining coverage gradually as automation is applied.

Every step in the graph involves a task.  but initially I recommend to automate against Mozilla Firefox and then continue with Chrome, IE, etc.
Next posts will describe each task in more depth.

Step 2: Customize your development process to provide support to automated testing.(Continued)

Tester Tasks:

Be part of planning meeting for current sprint: the tester will be in charge of provide estimations about data creation, test cases/acceptance test design, test execution, frameworks design/improvements, scripting tasks, setup environments, etc.

Be part of planning meeting for next sprint: the tester just needs to validate if he will create any complex environment before starting next sprint, or major improvements for the automation framework.

Write Acceptance Criteria for each item (*) for next sprint: Testers create acceptance criteria for the user stories compromised for next sprint, they work helping BA (if the team includes BAs) giving suggestions from QA perspective, Standards, User Experience, possible performance issues and future bugs detected. Depending how agile we are… we will have acceptance criteria and/or some test cases (also depending the company. sometimes you need to show some evidences about executions as test cases).

Create acceptance criteria could be a very creative task but we need to validate how possible they are and if we are in the same page that the PO, so a validation with the team could be necessary.

Update AC according to PO comments & Create Test Cases for next sprint: Once we received the team feedback we update the acceptance criteria and start creating test cases (or acceptance test if we are working with APIs).

Automate APIs (Current sprint): If the application is divided in tears, we will have APIs or Services. We could start scripting before programmers finish coding for the current sprint.

At first glance, all our tests will fail but gradually programmers finish their work our suite will pass.

Automate them is easier than UI and easier to maintain as well. Automation could be affronted using some tools like SoapUI, Jmeter, etc.

Execute AC manually (Current sprint): Test cases should be executed manually at least once when each User story is done. Manual verification will give us a set of bugs, an idea about limitations and how critical a test case is. On the other hand, this practice will determine the automate candidates for next sprint.

Automate Smoke Test UI / Regressions (Previous sprint): Automate UI once the application is stable could avoid you many headaches. In this stage, we will start automating the Smoke test for the current features done or regressions for complex modules. In feature iterations we will update the smoke test adding the new features. (see Step 1: Automation Feasibility validation. Run the checklist : Execution )

Exploratory Testing: this kind of testing will find bugs that no automated test will find ever. There is nothing compare than the tester creativity and we should apply this kind of  testing once every sprint.

Be part of Review meeting for current sprint: Testers or BA could lead the internal/external Demo. The goal is to show the commitment for current sprint and "ideally" we shouldn't show any issue live! So a Tester could choose the safer path to be shown and point out the known issues as well (an extra slide could be enough). Testers and BA speak human language, programmers don’t. ;)

 

Be part of Retrospective meeting for current sprint: In this meeting, everyone will have the same participation identifying things well done, things to improve and actions to apply.

Step 2: Customize your development process to provide support to automated testing.

TESTERS WORKING IN AN AGILE TEAM

The new tester role is more versatile and with a wide range of skills. Now, Testers have tasks along the sprint and they also program, yes!... as you heard, a tester programs as a common programmer (I don’t say developers as all participants in the team are part of the development so.. everyone is a developer: the BA, the Designer, the Tester, the Tech Writer and the Programmers).

New testers don’t need to be specialists in any language, the more languages they know, the better the solutions they will find. A good Engineer in testing should be able to adapt their knowledge according to the needs.

Programming Language Hall of Fame 

The hall of fame listing all "Programming Language of the Year" award winners is shown below. The award is given to the programming language that has the highest rise in ratings in a year. 

Let’s examine in greater depth the testers working in a team and their tasks distribution along the sprint.

(This process localizes tester tasks, it could be missing tasks for other roles).

Considering an Agile methodology, we could have previous, current and next sprints. Our Tester will have task related with each of these stages.

(To be continued...)

Step 1: Automation Feasibility validation. Run the checklist

As I said on the previous post, the first step is to validate automation feasibility. 
I have created this spreadsheet as a new artifact to validate Automation feasibility for software developments based on my previous experience in automation projects.

Download Automation feasibility checklist

Analysis

Why Automate?

-Better ROI
-More Coverage
-Reusability
-Less time to execute

When to Automate?

Project Inception validations: This is the most important validation we should do. A negative result here will conclude our process executing the checklist.

Sprints Validation: If we got "High Feasibility" for project inception analysis we will focus on the right moment to start automating throughout the project.

Which Automation tool?
We should research about existing tools and open source frameworks to make a decision:

Design & Construction

• Vertical or Horizontal Automation?
• Vertical: Automating End to End Flow where each test cases is independent of others
• Horizontal: Execute test cases features wisely to save execution time
• Keyword or Functional decomposition Driven?
• Keyword Driven: Each business process is mapped into actions and further each operation is mapped as a keyword. It is easy for non technical users to create test scenarios without knowing much of the testing tool.
• Functional Decomposition: Business processes are created first and while creating the test scenarios and test scripts, each business process is called in a sequence.

• The Data design:
• “Keep everything, that changes or has chances of changing, separate from the script.”

• Environments:
• Environment isolated and similar to production will guarantee reliable test results.

• Coding Standards:
• To apply specify coding practices and naming conventions will provide a good quality code.

Execution

What to Automate?

If you reached this point, you are ready to see all the benefits of your efforts. Mainly, we will identify the candidates for automation among test cases, based on:

Finally, This artifact will provide you evidences for rejecting/accepting test automation for your project and it will guide you in your choice of an automation tool , comparing the main features between them (Analysis section). Once you have decided to apply automation, Design & Construction section will help you to organize your framework following code and maintenance standards. And then, Execution section will identify key factors for test cases candidates for automation.

Hope you find this artifact helpful. If you have better ways to do it, by all means, do share with us.

Let's break the ice - To Automate or Not to Automate?

Test automation could be a key for success in software development. However an incorrect analysis of automation feasibility could result in unnecessary efforts by the Test Engineer.

Anyway, an unstable product will cause the test to not be able to be run. The test will not be able to be run manually either. So this case, completely rejects the possibility.

Feasibility will be determined according to the project schedule, technological impediments, and business value that the automation suite provides. If our feasibility analysis gives us good results, the next step should be to focalize in “Designing” a maintainable framework following international standards and patterns.

Key factors

Would you automate a product that takes 2-3 months to be developed?

Project schedule is a key for ROI. The number of times the tests will be run will be crucial in relation to the time invested in creating them.

Would you automate a product being developed in an obsolete technology?
Developments not within technological trends will conclude our analysis with negative results. Obsolete technologies don’t have the support of the community and it will be annoying to solve issues yourself.  

Would you automate any test cases that only you (the tester) will run?

Business Value refers to the benefit that a client (outsourced scenario) or the team receives of an automated bunch of test cases. In many cases, the automated test suite is part of the deliverable product and the client could use it in the future.

Would you automate the validation of the content of an image, or an exported PDF file that includes the information you need? Many times we face limitations that technology can not resolve. On the other hand, Human beings get bored running manual test cases but an automated test is not going to find new bugs. Human beings have creativity and some test must be done by a person. Is the application failing because we found bugs/regressions or because our framework is failing? An incorrect framework design will lead to false negatives, we should design a framework taking care in reporting and the right validations. From a QA perspective, if we get a low automation feasibility, it will make sense to focalize the QA effort in manual testing. Otherwise, the procedure to apply automation from scratch to a project will be:

1. Automation Feasibility validation. Run checklist.
2. Customize your development process to provide support to automated testing.
3. Tool choice. Use an existing one or create your own (language and libraries - i.e: Webdriver)
4. Isolated environments creation (Data and cleanup process).
5. Develop your test cases and run them.
6. Identify test cases candidates for automation.
7. Develop your automated scripts.
8. Execute your scripts against 1 environment.
9. Refact your scripts to give support to other environments.
10. Run test suite crossbrowsing (i.e Selenium grid)
11. Configure CI server
12. Report results.

This blog we will describe each point listed above.