Why do you even need a QA department?
This may be obvious to anyone who has worked in QA for awhile but it can become a murky, grayish area to others. There are some developers and company bean counters who do not fully believe in paying just for testing. And I get it, if the devs are doing their own testing and everything is passing with no big bugs are getting out into the wild, I can see why they may not think a separate QA installation is necessary. However, there needs to be a divorced perspective between coding and the final product. A tester is someone who can fully use the UI and who can match requirements to what is being delivered to the customer. They have certain skills that can maximize testing efficiency, meaning the most test coverage in a minimal amount of time. Another glorious benefit to having QA is that it leaves more time for developers to, well, develop.Why automation?
Going to keep this simple. A solid automation framework in place will help with regression testing which will help save time and money by having consistent tests catch any changed behavior to areas that were thought not to change. When you have most of your system automated QA can then spend their time on other areas that need human eyes on.Where to Start?
Sell the sizzle to the people in charge. Yes, there will be an upfront cost from manpower and tools to get it going but that will not only even out over time but should save the company money by reducing manual testing hours and ensuring a better quality product which will create stickiness for your customersOnce you have the green light you need to figure out what type of a framework will work for your product. If it is a traditional website using modern day browsers the popular Selenium tool would work just fine, especially if you are already using C# for your code.
Next, get with a willing developer and hash out your plan and see if they have any suggestions or concerns. At this point it should be established that the dev will need to have an initial time investment committed to helping the QA automation engineer so their time should be accounted for and expected. This would also be a good time to discuss test or behavior driven development. You will also want to find out what hardware you will need. Will you use in-house machines, VM's or some type of cloud option. You may want to have IT in on this discussion.
If you already have a ticket system in place like JIRA you should start your own automation project and create some initial tickets. This will inevitably grow to sickening proportions,
Side note about automation and JIRA tickets. The QA Automation engineer should attend regular ticket meetings. This will ensure that they have a voice when estimating tickets for any new code-work being worked on. If necessary, the work for the automated tests should either be added into the original ticket or a corresponding ticket should be created specifically for the automation work.
What types of tests to automate.
Functional Tests – These are tests that simulate customer input/output UI workflows like logging into the site, navigating to the billing page, changing info and then clicking the Save button. These are great for regression UI testing and will ensure the bulk of the site is functioning properly, however, these are time consuming to run. More on that later.Smoke Tests – These are great for hitting the UI and just verifying the page loads. This can be done by asserting a specific word that is present once the page loads. These run much quicker then functional tests but still take time since it uses the UI. These are usually run at the end of a deploy to the lower tier environment like Dev/Int or Test.
API Tests – These lightweight tests give you a lot of bang for your buck. These can be created in a REST client like Postman and integrated into your deployments. On the most basic level, you are just hitting a specific URL and making sure that you receive a 200 OK back. If you go a little deeper you can create tests within these tests that also verify specific words are returned. By bypassing the UI and hitting these areas, you can cut down on overall test time. These are also key to a Continuous Deployment environment as they create tight feedback loops which will identify issue quickly. These should also run with the smoke tests after each deploy.
Unit Tests – I mention these because they are a first wave of defense written by devs at the same time they write their code. These can sometimes be integrated into the main functional tests.
Security Tests – These can be tougher to automate. You're looking for weaknesses in the system. This usually takes human exploratory testing, but once you identify the areas you can then automate testing them as a form of regression testing.
Example of where tests fit into a deploy pipeline.
What Areas to Test First.
If you have no tests written at all you will want to identify the high risk areas first. If there is a sign up or activation process that ends with a financial transaction start there. It's probably a process that won't change too much and it also is what brings in the money. You always want to know that the area that brings in the money is working. It is also probably the first interaction your customer has with your site and you want that to be consistently working.Once you have the high risk areas covered you want to go through the next important actions which are time consuming to test. The more you can take off a manual tester's plate the better. These are areas that you can easily create functional tests for and you also want to make sure these are areas that won't change much. You do not want to invest lots of time that has a redesign already planned in the near future.
These types of UI based functional tests are good examples of what should automatically be run at night, after business hours. Depending on how big your test suite is it could take over an hour to run them. Running them at night helps avoid systems going down due to other testing going on. Then, when you come to work in the morning you will have a suite of results waiting for you to dig into.
Comments
Post a Comment