I lead Open Source and Developer Advocacy at BrowserStack, working at the intersection of engineering, community, and product. I am a core committer and Technical Leadership Committee member for Selenium, where I collaborate closely with browser vendors to advance web automation. Over the past year as a Developer Advocate, I have been deeply involved in enabling and empowering the testing and developer community through meetups, conferences, blogs, and hands-on how-to videos. My work focuses on improving open-source tooling, contributing bug fixes and features, shaping best practices for test automation, and helping teams adopt modern testing approaches using BrowserStack.
Speech: Making AI-Generated Browser Tests CI-Stable: Lessons from Selenium
Selenium doesn’t mock its own tests. It executes them against real browser and OS combinations. This exhaustive approach ensures the framework is reliable before code merges. When a test fails on CI(Continuous Integration), debugging it teaches you how the browser automation ecosystem actually behaves.
Over years contributing to Selenium, certain failure patterns emerged consistently: browser environment mismatches, cross-browser behaviour differences, spec interpretation gaps across browser vendors, inconsistencies between headless and headed modes, element synchronization issues (waiting for elements to be present, visible, or interactable), CI-specific constraints (CPU, memory, network variability), and test isolation failure.
These problems aren’t new. But AI-assisted testing makes them common again since the implicit context that comes from real debugging is missing. As a result, generative AI churns out tests that pass locally but fail in CI. When the CI error gets pasted back to AI, it regenerates code without understanding the root cause, often sending things in a different or frustrating direction. AI isn’t at fault, it’s simply skipping the learning journey developers take when debugging.
This talk breaks down each failure category (using real test cases from Selenium code-base) and surfaces the implicit knowledge behind them, the kind that usually takes years to build. With this, the patterns become recognizable almost immediately and applicable to your own tests. More importantly, you will learn how to bring this context into AI-assisted testing so the same failures don’t keep resurfacing.
Key takeaways
- Learn the common failure modes in web browser automation, especially those that surface in CI.
- Understand how these failures are identified, diagnosed, and resolved in practice.
- Integrate these learnings into AI-assisted testing to reduce churn and improve test reliability.
