This page was machine-translated from Portuguese and may contain mistakes. Read the original
In this Tech Chat, I was joined by Stefan. He has long been a leading figure in the field of quality assurance. He has spoken at numerous events and has contributed greatly to the IT community in Brazil. And this time, he tells us about his extensive experience with Playwright and his perspective on the tool since its launch.
Stefan, thank you so much for your contribution to the Tech Chat. I never stop learning from you!
1. Tell us a little about yourself and the projects you’re working on with Playwright.
My name is Stefan Teixeira. I’ve been working in QA for over 13 years and serve as a Quality Architect in one of the departments at my company. I manage multiple teams within the department, working on various web projects using JavaScript, TypeScript, and Ruby on Rails. I’m responsible for the department’s QA and CI vision and strategy, leading the 15+ QA professionals who make up the teams and working closely with the Engineering and Product Directors.
2. When did you start working with Playwright, and what was the decision-making process behind choosing this framework? And what language do you use in your tests?
In early 2020, I decided to use Puppeteer when we began implementing end-to-end tests in our main project. At the time, I saw that Playwright was an option (and that it was a fork of Puppeteer), but it had only recently been created, so I decided not to take the risk.
In early 2021, I noticed that the jest-puppeteer integration was no longer being maintained regularly, which was a major risk and made it impossible for us to easily use newer versions of Puppeteer. So, I migrated the tests to Playwright, and we’ve been using it in all our JS/TS projects ever since.
3. Do you have experience with other frameworks? In what ways does Playwright generally stand out?
I’ve been working with test automation at multiple levels for at least 12 years; throughout my career, I’ve worked with various frameworks in different languages, such as Selenium WebDriver (and RC), Cypress, Protractor, etc. I can confidently say that Playwright is the best framework available today and is far ahead of other options.
Playwright’s native parallelism and its speed/execution time are key factors and definitely its strongest points, in my opinion. Fast feedback is crucial during development, and with Playwright, we can easily parallelize tests according to available hardware resources.
4. Is there anything you think still isn’t quite right with Playwright?
Playwright has come a long way in terms of debugging features recently—an area that really could have been better. Currently, the feature I’m hoping will become more robust is Component Tests. In front-end projects, it would be great to be able to replace React Testing Library with Playwright for this level of testing, which would allow us to run tests using real browsers. However, this feature is still experimental, and I hope we’ll see more updates in the future.
5. What is your favorite feature of Playwright?
It’s definitely the native and easily customizable parallelism. This makes it simple to scale execution and has a huge impact on the development of our projects.
6. Can you tell us a little about how your pipeline is structured? Do you use parallelism? What about sharding?
Tests always run in the pipeline after a pull request is merged and must pass for a deployment to occur automatically. Depending on the project’s size, we run tests on pull requests manually by adding a comment that triggers the CI job. We always use parallelism, but we have our own script to perform a process equivalent to Playwright’s sharding. For very large projects with a high number of tests, we use this to distribute execution across multiple machines.
To handle the increase in the number of tests and their distribution across different machines, we must evaluate four aspects: execution time, level of parallelism, level of distribution (sharding), and CI costs. Execution time is important, but we need to consider the costs when thinking about adding more machines to run the tests.
7. What IDE do you use to develop tests? And do you use the command line more often, or the Playwright plugin in the IDE?
I use Visual Studio Code as my IDE and usually run tests directly from the terminal; I don’t use any other plugins.
8. What do you think of Playwright’s Codegen?
I haven’t used it myself, but a while back I saw that a developer on one of the teams was using it and liked the experience.
9. Have you ever used GitHub Copilot to implement tests? And have you used artificial intelligence to create any tests or functions (such as a page object model) in your project? Do you have any thoughts on these topics?
I haven’t used Copilot yet; I’ve only used ChatGPT (GPT-4) for general purposes. Using AI makes things a lot easier and helps speed up the development process, but before using it in a work environment, always check which tools are approved by the company and be careful not to include sensitive or confidential data in your prompts. There are still many privacy and information security issues related to the use of generative AI that need to be carefully analyzed.
10. How do you handle framework updates? Do you update automatically or manually? And do you have any difficulty dealing with the frequent number of releases? How do you determine/prioritize what will be updated in the existing code with the framework’s new features?
For smaller projects, it’s simpler to update versions automatically (using Dependabot, for example) and verify that the tests pass. For larger and/or legacy projects, you need to pay closer attention and also verify that the application is functioning correctly. My recommendation is to use Dependabot and adjust its configuration after coordinating with the team on the frequency of dependency updates. It’s also important to create tasks for these updates, which should be estimated by the team and prioritized in the backlog.
As for code updates with new features, you need to review the release notes and see if there are any features that would be useful for the project. If so, I recommend following the same process of creating task(s) in the Backlog, estimated by the entire team and prioritized appropriately.
11. Besides UI and API testing, do you use Playwright for any other types of testing?
I use Playwright for different types of testing. Mostly for end-to-end tests, visual regression tests (integrated with Happo.io), and integration tests (checking only the application’s front end, testing responsive design behaviors, simple workflows, etc.). In addition, we also use it for more specific tests, such as accessibility testing, SEO testing (validating server-side rendering—SSR, validating meta tags, structured data schemas, and open graph tags), and API testing.
12. Do developers also maintain the tests? If so, how do you ensure development standards when professionals from different backgrounds are working on the same code?
Certainly, this is essential. I still see many companies and professionals with a culture that separates “QA testing” from “developer testing.” This mindset is completely outdated and prevents us from having better tests at every level. In a team’s day-to-day work, it’s natural for each role to handle specific levels of testing. However, it’s crucial that all team members know how to maintain all levels of testing and are able to analyze logs, debug, and identify and implement improvements.
As for development standards, these are ensured through the use of linters and code formatters, such as ESLint and Prettier in the JS/TS world. It is recommended that teams create their own custom rules to ensure that standards and conventions are followed in the code. Finally, it is also important that pull requests be reviewed by different team members (for example, 1 developer and 1 QA). During these PR reviews, team members can identify improvements to linter configurations and/or ideas for new rules to be created.
Additional References:
- Stefan Teixeira’s Talk on Playwright for the Minas Testing Conference 2022
- Other Tech Chats
- Playwright Training Sessions by Renata Andrade
