End-to-End Test Automation Without the Maintenance Trap - Lilia Gargouri

0:00Test automation is software development. It's a software project within the big one. Definitely. Do not underestimate test automation. So I don't have 500 test cases. I am not the fan. I have maybe 15. But they really cover I I don't know 80% of the m most things It's like if you search for something at home. It means your system is not good enough. Create a parallel one. This is the old word, you will have the new word. Welcome. Welcome to Software Testing Unleashed.

0:38My name is Richie and I am happy to share another episode from the TestCon 5 conference from Mauritius with you. My guest today was Lilia Gargouri. We talked about end-to-end Test automation at scale. How one small HTML attribute from 2017 still pays off today, why clean code and naming rules save you from the maintenance trap, and what to do with an old flaky test suite. That all you hear now in this episode and enjoy it. Hi Lilia, great to have you on the show here.

1:21Richie is so nice being here. Thank you so much for inviting me. Yeah And bringing topics at every conference basically? Yes, it's uh I I think I'm on every conference now. No, but I I I I love it to share uh insights with the community and uh we are here on Mauritius and no not everybody is able to be here at the conference so with this podcast we have a a little opportunity to give some of the knowledge and the content also to the community worldwide.

1:50So that's always a good thing. Unfortunately you didn't have the minister Yeah, he talked really brilliantly today about ethics and software device. Yeah, yeah. I like it a lot. And I was really I was really fascinated that uh politician minister has so deep knowledge into uh IT and technology. I tell you the why he was software developer. You mentioned this so this is the thing, yeah. Yeah, yeah, great. Uh uh yeah I I think other countries can Can look at that and so yeah but but now we want to talk about uh your topic you also presented here on uh on the testcon conference.

2:26Uh uh and it's about uh It would I would say modern state-of-the-art end-to-end testing. How do we do it nowadays? Absolutely. It's about end-to-end testing at scale. Scaling automation uh and especially end to end so you are on the big peak of the test pyramid this is a lot of flakiness and if you ask people how many time you spend in automation for what They will say forty percent flakiness and maintenance. Yeah. It's not fun. So you get very quickly in the maintenance trap.

3:01The same problem. So I am not bringing New problems. Yeah. But the thing is we are still struggling with these problems, and if you you use AI, you are augmenting very fast the problems as well. Yeah. So it's not about generating very fast the code. It's how can I scale this? Yeah. And you know in in my company we do automation for big projects in e-government. So it's enterprise software, which means you deal with very high complexity of of business.

3:35Plus it's a long life application in the terms of 25 years old, changing with cycles, releases, and you have maintenance of existing things, but also evolving things So is scaling, even the number of your tests is scaling. So your test execution time is scaling, your maintenance is scaling. How can you deal with this? So it's really struggling with this and many people they come very fast into many, many problems. This is because you start within without having a clear strategy.

4:12Uh what what can a clear strategy be for such an environment? I like this question. Yeah, I hope The first first of all before many people start already generating the tests and generate implementation and whatever yeah is not The the first thing is and because we talk about UI and in my presentation it was web UI because Uh internationally 90 to uh 80 to 90 percent of the applications are still web UI, so it's the most automation is web UI worldwide.

4:47And the first thing to talk about is how can I recognize accurately every single smart component on the UI of my application. So you have the HTML page or whatever. How can I accurately target one small tiny thing. And now it can happen that this tiny thing is Icon button. It does it's a picture inside and it's very small. It's the garbage, you know, you know the delete button. But you have fifty Delete buttons in the table. Now target the specific one you want.

5:22So there are many skills, first of all, to to Improve the testability. The first thing we did, and this is something I can really recommend to anyone, is first of all improve the HTML, the quality of your HTML. Otherwise it's really pain every time. It's like you solve the problem just temporarily, but it's not a permanent solution. So the solution is To increase the testability, we added an HTML attribute only for test purposes. We call it data rule attribute, you can call it test ID or for testing purposes to attribute.

5:59Yeah. And you give it a value. And this attribute is only for testing purposes. It is a must-have required if it's missing you uh raise a buck ticket for the developer so it's a must have then consistent over your all your application and never change yeah so it's always there and now you can start building sustainable stable uh recognition. So what we did is first of all we introduced this attribute. This was the decision 20 17. You see how we are reaping the benefits of this decision?

6:36Automation is about very tiny decisions. That bring efficiency in sum together. So one of them was the data role attribute to improve the testability. The second thing now I have data role attribute. I start mapping them to the Test automation class so that this is a recognized this attribute value is recognized to a table, to a row, to a cell, to I don't know, icon button, info button, I don't know This is the second thing and the the piece of s the mapper, the piece of software that doing the mapping, we centralize this on one place.

7:13So that you have the maintenance, the implementation is central. And if you open the browser immediately after opening the browser you have the mapper. So now you can have a lot of test cases everywhere in every test suite. Your tool recognize accurately every element. This is the first thing. Secondly, we improve this small tiny solution with some small script where we use the attribute like area label alt attribute of accessibility because we have a lot of a government So we test implicitly even accessibility.

7:50It's one ta one shot is done manually so that the value of the accessibility attributes are meanful to the blind people or others And then in our scripts we are extracting this valuable information to enrich the identity of the UI elements. It's brilliant basically because if you read the code you know what's about. Yeah, because it's for the blind people is also for me as very meaningful. So this is the second thing. uh a text field name and you have many sections and the name field existing in many sections so now I want that one the fifth one yeah so the thing is and and this is also a brilliant decision is to say I have a big HTML Test tool for you, only this small session exists.

8:46The DOM doesn't exist. It's just this piece of thing. It's the HTML that exists for you. So the rest is gone. So now in this, so this is the HTML page now for the yeah. So for you this section and now I can target the UI element. Now because you remove the lot You can target accurately your stuff as small. If you make it small, your scope is called the scope. You can target very now, very precisely. as because it's so small. Now you you can accurately and and now you get stable stable runs.

9:22You can make reruns and you rely on it. It's not flaky Because the combination of all of this together and the one of the other decisions is You can make it generic. You can make the labels, so we use identification by label, so text field, so it's a text field with the label name. So we t it uh you can say okay it's not a good approach for us for a government it is good because they don't change the head the labels of the input fields. So it's stable.

9:53Maybe one time, two times, but that's it It's really stable. So we I target the by name so I can read the code easily and analyze the code easily. This is the thing. And the other thing is You can if your application is multilanguage, you can put this in an Excel file and say Italian, English, I don't know, and you run the idea is generic. uh dollar label and get the label from there depending on my login I can get the right language. The nice thing is It's the same code, is one line it's the same ID, the same line of code is running on diff with different language.

10:31So you don't duplicate the code at all. So you see there are many generic things making identification brilliant. Right? This is just one aspect. Now we can dive deeper if you want, but this is really the beginning. Yeah, I I like I like the idea because it has Nothing to do with all the AI stuff and and so it's it's it's it's really uh solving a problem we we have in end-to-end test automation since years now and we often pack things above it and try to solve it but but this are real uh solutions for in in uh to make it more testable and and maintainable yeah.

11:07Yes and and and you know there are people still using XPath and whatever and or ideas that are concatenated and then in maintenance so if this line of code fails you have first to split them say which one Is and now imaginate you are in the testing phase and you run out of time and you are under pressure already and you have first to analyze is it bug or feature? So you have to split this concatenated IDs to know which one. The s sadness is you have to repeat this each time.

11:40This line of code is failing. This is crazy, right? Why you do this? Yeah, so true. Yeah. front uh and and and then so what's what's the structure okay so so the second so this is the first thing we didn't start implementation yet right we did just the recognition brilliant the second step is Still not the automation. The second step is assess and define. Design your test cases and think Because it's not about quick quick quick and yeah, it's I need only the criticals, the very very valuable scenarios that make a difference.

12:30So I don't have five hundred test cases. I am not the fan. I have maybe fifteen. But they really cover I I don't know eighty percent of the M most things so if they are green we can talk about the rest. So you see it. So this is a strategical decision. So now you have to say uh and this is what I present there are three pillars that assess the quality of one test case. It's like you know you go a journey and you have a backpack and you go for uh walking for a long time, so it's if it's heavy you have a problem, right?

13:05And a test case is one stone and you are packing this in your backpack. And I have to walk for over years because it's a long life application. So I have to decide, do I want this? You know, some people have refrigerators in the backpack. Do you want this? So the first pillar of a test case is the scenario. What do I need this? Do I need this? What is the focus? Sometimes you don't even have a clear focus, so clarify the focus. What what what is it critical?

13:39Um you have steps in your test case. Do I need this step? If I remove it, is it still the same uh coverage? The same scenario and philosophy? Where So minimize to the minimum so it's it's not blindly blindly taking from the test management tool and auto making automation and redundancy and everything. I don't do this. So we assess this and we squeeze this to the what is the core essence of this test case. This is first the scenario. The second part is You know, you can have a brilliant scenario, but you use, and I hate this, people are using test data ABC123, everywhere.

14:18No value. You know, very expensive automation without any value. And this is uh uh boundary uh you know for ICOB Foundation level. So What which boundary should I use? What equivalence classes? Why I need this. So think about the test data you are using in this expensive test scales and choose the one the best ones and and with the target because you have to c cover something with this once and then are they reusable? How can I generate them? In a sustainable way.

14:52It should scale and it should survive and the maintainability is also the test data. And then the third thing is the implementation. Now we can say no, I do it manually. So if you do it manually And you miss one step by you change the philosophy of the test case. You change the coverage, you change everything. If you do it on automation, now we talk about automation implementation, how to implement it in a way that the maintenance is low, that the this automation nevertheless is low and this stuff.

15:22Now we go to the next chapter of this talk. Yeah, yeah. Yeah. So this was And then after defining now I treat you so hard in to evaluate which test case. Now let's say you define your backlog. Yeah Still, we don't start. Please do not. Now assess I can prioritize my backlog by depending on my strategy, my situation, and the project, I can say, okay, the business criticals first. I take them first and automate them very quickly so that I can see them and if Jenkins is green, I am good at least for the criticals.

16:00I can say no no no no I pick up the test cases where manual test if they do this manually it takes long time if I do this I relieve my manual tester team And then they can focus on new features or something. This is a better strategy, maybe. You can say, okay, I cover the happy paths, or you can say I cover the most used, I don't know. So there are many, many factors. That can change your mind and it depends on the here and now and your strategy.

16:29So the backlog is not one two hundred, so I go forward. Yeah, so this is also the thing. And then you start implementation. Now you have ev everything done b on purpose. Yeah. Right? Yeah, a great foundation for for then implementing the test cases. Absolutely. And now you The first thing I said after this, so now we start. So software develop so uh test automation is And I am software developer. I did 15 years software development before moving to quality engineering and automation at scale.

17:0412 years. So on top. So I can tell Test automation is software development. It's a software project within the big one. Definitely. Do not underestimate test automation. So Software, because it is software, it has also architecture. So you need a framework, architecture So the framework we we use is really by chance I didn't know about it, but identical to IST Cobetes Automation Advance. is you have three layers. So you have the core libraries and then you have the business logic and on the top you have the test scripts which means On the top you have the test cases and the test suites and the test cases call the procedures which are handling the business.

17:54And the business logic and the business logic is calling the libraries. And regarding the libraries, we did Many layers of libraries. One is straightforward and is given by the TIS automation tool. They have their own standard library. So it's not enough for me. We build a library cross-project So that not every project re invent the wheel and you know you there are functionalities like browser switch up Why should we implement this in every project everywhere?

18:26It's a library, the company library for automation, for example. Then we have the project library where for example you have the context menu It's everywhere the same UI in the application. You click the three dots, it opens the context menu, and then you choose select the action. One procedure doing this, it just give her I want the action blah and it's doing this for you. Yeah. Why everyone Making a test case have to implement, you know, you see.

18:56And and and then you now you have a good foundation with libraries. Every time you see that you are inventing something new, put it in the right library And then you have a nice thing. Email library for testing emails, PDF library for testing the PDF content, so that you prepare first a solid foundation to be used. Why? Because then you accelerate and facilitate the automation, not not only for one project, for all the projects within the company, yeah, depending on on the thing Yeah, this is so first the framework and then now we go further and say you have a clear structure, clean, clear structure, naming convention.

19:33Here at the naming convention we do Uh so I invented the convention. You can invent your own convention. The important thing is use it everywhere. Yeah All time. Is your convention is fine, but use it. This is because you guarantee the clean code everywhere. And and my convention is simple. It's action and associated with business. So the action is enter or check because you have input fields so that is enter something in the input field or check the value of the input field.

20:07So enter or check or click if you have um buttons on the footer or on the header so action and then the business it means the label so enter name is you are putting something in the text field name or whatever the the technical identification of the UI is not part of my wording. I don't care. It can be a checkbox, it's enter Yeah. Don't tell me. It can be a radio button, but it's enter. It can be a date picker and it's enter. Yeah. And it's not enter date picker, it's enter and the label of this date picker.

20:41I don't know. It can be from to sometimes, yeah? Yeah. So it's I know my application. If I close my eyes and can navigate through my application. So yeah, my application with my business. So if I read later the code I understand where I am and what I am doing. Straightforward. You don't need to if you have new colleagues and I um there is a study by Red Code, they are telling you to onboard a new colleague. You can decrease the time four times lower.

21:15So it's multiplied by four if you if you have clean code You don't need to to bring doctor many. So the structure in what we do in automation is exactly the same as on the UI. So you can navigate in your UI, you can navigate in my code. And then because we use this naming convention, you can read what we are doing. So you if you Top yourself searching for something. Where did we implement? Did we implement this? And where? I don't ask this question.

21:46If you tap yourself doing this, is you are missing Classification structure something. It means your system is not good enough Yeah. So so how to structure the things? So structure is really because it has to scale, it has to survive, it have to be for a long time. You have new colleagues coming in and out all the time and and things. So you And if at the end of the story in the test case, because you call the procedures, right, you can read what you you are going because it's meanful.

22:24You can understand what what we are doing in this test case. Very well. Absolutely. And the thing is, yes, it is a discipline. But it's meanful and bringing a lot of efficiency. So if someone joined my team for two weeks or two months, sometimes two months, depending on the person you are still working on one single test case. Why? Till you absorb the naming convention and you absorb that you have to use it everywhere and and and and and you stick to this you know, you don't make the rules just for fun.

23:03It's really because they bring the value at the end of the story, yeah. It's it's many tiny small things, but together they bring this, you know, th this efficiency, this easiness to analyze the test report. This was one of the chapter of this presentation. So just to close the implementation, design pattern, design principles is is software development and you have to use it also in automation. So yeah. So a lot of this and the chapter after it sh should I continue great flow here.

23:37So af after this uh I I brought the chapter of My application is a long-life one, so the number of test cases is scaling. Automatically the test execution time is scaling. And the maintenance, we said we have clean code and so on, so the maintenance is really low. Why? Because we have modularization everywhere. Centralization everywhere, generic stuff, so that if something change, I have maybe if I am lucky one place. If you search for an ID or something and you find five hundred findings.

24:17This is a ticking bomb. Why? They changed something and you have a release candidate. You have a rough time. For testing and now this changed and is not a bug, it's a feature. So now maintain 500 findings very quickly and make them working again. is so yeah it's it's it's really bringing a value uh it's because it's critical. Yeah. Okay. And now let's talk about how to make the execution. So we talked about uh the identification of the components.

24:50This brings the stability in the execution. Um there is also busy uh application busy detector. This we build it also on top of this resolver or the mapper. So application busy as long as you see the spinner you wait. This is not the sleep time because the sleep time is accumulating. Uh we analyzed one by one project. We had four 40 test suites, only the sleep time, you know. You see just two milliseconds here, two milliseconds here. You do it was two hours in total for one run.

25:25Yeah, losing time for nothing, you see? So it's it's really important to keep an eye on this. Don't say mm, I have flakiness sleep time is helpful, go for it. It's not. Because you don't solve the problem at the root of the problem. You are just making, you know, it's like you have an injury, you just put something quick. on on this just temporarily basically and this is one one point and and then to bring the stability you have also to make your test environment always up to date You know, you are dealing with a lot of software versions.

26:01The browser version, the Java version, the Gradle version, you know, it's all impacting you your uh system under test version uh uh your test automation solution and the test tool is changing everything and they have to how how to say they have to gather To be stable by working together. And then the last thing is if you start your test cases, the run, the test run. Make sure that there is a in this environment, it can be a virtual machine for example, or container, or I don't know, that there are no other jobs running on here in parallel.

26:41Why? You will have side effects. And problems that are not your problems. And you get crazy. Why? Because it is impacting you and you don't know what happens. And you are guessing now is it a bug in my application or something weird? It can influence also the performance. And then the it's not working smoothly. So make sure that you don't have this and and another thing is Another process is changing exactly the same data you are testing with. This is hard because you get exceptions and stuff.

27:16So make sure that you don't have this And then uh to bring uh flexibility in the test runs, we did modularization so modularization in the court. Now modularization in the meaning of each test case is a unit and it's modular and independent so before me is after me I don't care so you can put it everywhere You can you can bring things together without having big deals. What does it mean? If you have make an pact analysis, I say I need this topic, this topic, this I can bring them together in a test run.

27:53Why? Because I am modular. And the test suites are modular as well and they are designed or separated by topic. So business. If a test suite gets longer, takes time and they run longer than one hour. One hour is now my limit. You can define your limits, say 30 minutes or five minutes. I don't know each one. So because my test cases are modular, I can split it easily into subtopics. I don't care. So I bring flexibility. And then I can now strategically in my situation say, okay, I run the smoke test first and then the integration test and then I go in the function.

28:30If this one is not working, I don't need to go further. I stop here and fix the criticals or I don't know. So you have the control over the things you can Depending on the situation can have the weird situations. For example, I have a business application runs on many techno so on electron and web and So uh the electron with the test automation tool they did an update so it's not working now. So I cannot run this test. So we cannot go live This is electron, I can run them on what the same test, the same code.

29:08It's just the other technology. I cannot run them It it's just the precondition changed by open browser instead of open this electron application. But the rest the test cases are modular. It took us five minutes to run the web tests and we had results functional results even not only integration till they fixed the the problem with electron. So you see you can you can bring flexibility Yes. It's that's a great uh flow through the through the test automation and I I uh how we should do it in the in the in the best case.

29:43Uh and uh I I think it it's it sounds so clear, uh but but often companies just don't do it on on any any things. But what would you suggest uh or what do you have for a tip for someone who has now an old N2N test suite there with Flaky tests with not a good structure with not good identifiers is a where to start and how because It's all uh a big system and uh where what will is your tip to start in such a situation? And it happens, yes. And if if I would join your team or if I I tell you okay How to start?

30:23First of all, separate. So this is your old old project. It's called test automation solution. So it's your git your repo. Let's say your repo you're in git. Create a parallel one. This is the old word, you will have the new word. Some at at one point you will you will just you know switch totally. and start the clean first of all the clean structure in a repo is not by mistake you have a clean structure of the template of the project and start here implementing the new test cases in the new structure with the naming convention and building the things.

31:03And now you can tell me okay but I need the procedures and the library Pick up the valuable ones from your old thing. And by picking this way, I took over as software developer in a project, the automation. It was exactly the situation. And now in the mess Pick one by one the the valuable things and build your library and build your core library and build your business things and build them, enhance them, improve them in clean code. And then you build Okay, so slightly then your best solution.

31:39You might write them from the old world to the new world. It will take your time, but You know there is always a beginning. And if you want to walk ten kilometers, you will start by one step. And if you don't do it, you will not come to the ten kilometers. You can forget about it and still um uh how to say get angry every day about your flakiness and and uh waste forty percent of your working time and and this is a quality of la life in maintenance.

32:12I I'm not sure about this. So so it's a quality of life I would say. That's a great point. And then I think we have to work for Lilia, thank you so much that you were here in the podcast. Uh a great way I think there are so many impulses and insights there we can now take in our job. And hopefully a lot of people get their end-to-end tests now better than before. Thank you so much. Have a great conference. Thank you so much. See you soon.