How do we evaluate photo-enhancing features on users' own content?
Scope the problem
To adequately determine how users view Google Photos features, it’s important users see how these features affect their own photos. We needed a way to a) apply unreleased features to users’ own content b) scale this method to collect quantitative feedback on user satisfaction and c) collect qualitative feedback on where improvements could be made to the feature.
Process
I partnered with our vendor, product managers, engineers, and other user experience researchers to develop a pipeline where we could conduct unmoderated diary studies that offered both qualitative and quantitative feedback on the quality of unreleased photos features. I drove this project from end-to-end, identifying technical pain points and eventually testing the pipeline on a currently unreleased feature.
Impact
This program has allowed Google Photos to increase the number of users in diary studies by a factor of 5. It also allows stakeholders to get a much more accurate perception of their features since the features are being applied to users’ actual content.