Machine studying fashions have been educated to foretell semantic details about person interfaces (UIs) to make apps extra accessible, and simpler to check and automate. At present, most fashions depend on datasets which are collected and labeled by human crowd-workers, a course of that’s pricey and surprisingly error-prone for sure duties. For instance, it’s potential to guess if a UI factor is “tappable” from a screenshot (i.e., based mostly on visible signifiers) or from doubtlessly unreliable metadata (e.g., a view hierarchy), however one strategy to know for sure is to programmatically faucet the UI factor and observe the consequences. We constructed the Endless UI Learner, an app crawler that robotically installs actual apps from a cellular app retailer and crawls them to find new and difficult coaching examples to study from. The Endless UI Learner has crawled for greater than 5,000 gadget hours, performing over half one million actions on 6,000 apps to coach three pc imaginative and prescient fashions for tappability prediction, draggability prediction, and, display similarity.