Performance testing is easier and more important than ever, especially with the recent emergence of Macro/Microbenchmark, JankStats, and other libraries/tools. While these methods provide a great way to catch regressions before they make it into production, determining what to test, how to create good tests to catch that “what”, and orchestrating tests at scale in CI introduces many new challenges to the mix.
I plan to give an intermediate-level talk about how to solve those new challenges when performance testing in CI.
I plan for audience members to leave my talk learning:
– What mobile performance metrics are worthy of being tested on mobile in CI vs what’s best suited for using production data.
– What makes for a truly useful performance test. (controllable, user-impacting, reproducible, i.e. doesn’t deal with network/IO latency, focuses on rendering performance and on-device functionality)
– How to build those truly useful tests to benchmark the mobile-specific metrics we determined we wanted to measure (leveraging Macrobenchmark/UiAutomator).
– How to combine all the previous points to orchestrate consistent and accurate tests on both emulators and real, physical devices in CI.