Agricultural pests and diseases pose a serious threat to crop yield and quality, making accurate and efficient detection and identification of pests and diseases crucial in agricultural production. In this paper, we propose a comprehensive agricultural pests and diseases dataset, which includes agricultural pest detection dataset, agricultural disease detection dataset, agricultural disease classification dataset, and rice phenotype segmentation dataset. By collecting and curating data from public sources and academic papers, we ensured the diversity and representativeness of the dataset. Rigorous quality control and validation measures were implemented during the data filtering, cleaning, and annotation processes to ensure the accuracy and reliability of the dataset. This dataset can be used for agricultural pest and disease recognition, as well as rice phenotype identification and other agricultural visual tasks. It provides valuable resources for agricultural pest and disease research and contributes to the sustainable development of agricultural production.