Cross-device federated studying is an rising machine studying (ML) paradigm the place a big inhabitants of gadgets collectively prepare an ML mannequin whereas the information stays on the gadgets. This analysis subject has a singular set of sensible challenges, and to systematically make advances, new datasets curated to be suitable with this paradigm are wanted. Present federated studying benchmarks within the picture area don’t precisely seize the size and heterogeneity of many real-world use circumstances. We introduce FLAIR, a difficult large-scale annotated picture dataset for multi-label classification appropriate for federated studying. FLAIR has 429,078 photographs from 51,414 Flickr customers and captures lots of the intricacies usually encountered in federated studying, similar to heterogeneous person knowledge and a long-tailed label distribution. We implement a number of baselines in numerous studying setups for various duties on this dataset. We consider FLAIR can function a difficult benchmark for advancing the state-of-the artwork in federated studying.