Impact Of Responsive Web Design On GUI Based functional Tests

I have been working on a new project which is about building a public facing retail website. One of the requirements of the website is to make it responsive so as to cater to the growing number of requests from mobile devices. As an enterprise if you don't have a dedicated Android or iPhone app, probably making a responsive website is the cheapest way to get your customers happy and served.

So not deviating from the actual point, we have a responsive web application in making. And as part of testing we have added a bunch of functional selenium tests. These tests are added to the CI pipeline and runs with every check in.

The QA team has a remote machine which is Win7 and is being used to test the application as well as author the functional tests. What we noticed that the functional tests used to pass locally on that Win7 box, however same test when checked-in was failing on CI agent. The log used to show "element is not currently visible and so may not be interacted with". While analysing we used to think that the page load is taking time, so we tried all possible ways of waiting and explored brilliant ways of waiting added in selenium as part of "ExpectedConditions" class. You guys should definitely explore that instead of age old "Thread.sleep".

So we were quite clueless as to why this is happening. We even tried logging into that build agent and manually executed that functional test gradle task, but the tests passed there surprisingly. And when we triggered the test pipeline in CI it failed.

We spent around a day investigating that issue, then one of my colleague guessed that it might be happening because of screen resolution and our "responsive" website. 

We ran the functional test reducing the screen size and the test failed showing the same error message. When we looked into the code we were actually maximising the browser (using the code driver.manage().window().maximize()). We found that the browser was actually maximising when we were running the test.

Then we started debugging the issue (putting some SysOut & playing around with the browser dimensions and all). When we printed the browser resolution on the CI agent it had a screen resolution of 800x600. The same agent when we log in via remote desktop connection had a higher resolution. So we found the culprit.

For that resolution, the application had to render an HTML suited for tablets and that had the navigation links hidden. Once the issue was found it was easy to fix.

A quick learning, do keep in mind the "responsive web" part while authoring GUI based functional test. The page navigation would change when the screen size varies and the functional test should handle it.