CLIP-RSICD: A Fine-Tuned Vision Transformer for Urban Land Cover Classification
DOI:
https://doi.org/10.5433/1679-0375.2026.v47.54490Keywords:
urban morphology, decision-support, spatial mapping, vision-language modelsAbstract
Reliable mapping of intra-urban land-cover categories at fine spatial and thematic resolutions is fundamental to evidence-based spatial planning. Nevertheless, transferring classification models across territories remains difficult because urban morphology varies regionally. This challenge is pronounced in South American urban centers, where rapid expansion alters metropolitan dynamics, demanding finegrained land monitoring to support zoning policies, infrastructure distribution, and ecological protection. Addressing this issue, we evaluate a remote-sensing (RS) adapted Contrastive Language–Image Pre-Training (CLIP) visual encoder for multi-class urban land cover (ULC) categorization. The proposed taxonomy incorporates ten classes, spanning high-, medium-, and low-density built environments, industrial complexes, vegetative covers, and exposed soil. The high-resolution (HR) ULC products offer a rigorous empirical framework for urban morphological assessment and planning applications in intermediate Latin American cities. Using HR, open-access satellite imagery, the framework was implemented across the central urban cluster of the Londrina Metropolitan Region, Southern Brazil. The domain-adapted CLIP model reached an overall accuracy of 93%, outperforming nine baseline deep learning architectures. Our findings highlight the capacity of vision-language backbones to capture spatial patterns and extract discriminative representations from RS data, providing a scalable solution for urban spatial analysis.
Downloads
References
Alem, A., & Kumar, S. (2022). Transfer learning models for land cover and land use classification in remote sensing images. Applied Artificial Intelligence, 36(1), 2014192. https://doi.org/10.1080/08839514.2021.2014192
Arutiunian, A., Vidhani, D., Venkatesh, G., Bhaskar, M., Ghosh, R., & Pal, S. (2021, October). Fine tuning CLIP with remote sensing (satellite) images and captions. Hugging Face. https://huggingface.co/blog/fine-tune-clip-rsicd
Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32. https://doi.org/10.1023/A:1010933404324
Brown, C. F., Brumby, S. P., Guzder-Williams, B., Birch, T., Hyde, S. B., Mazzariello, J., Czerwinski, W., Pasquarella, V. J., Haertel, R., Ilyushchenko, S., Schwehr, K., Weisse, M., Stolle, F., Hanson, C., Guinan, O., Moore, R., & Tait, A. M. (2022). Dynamic World, near real-time global 10 m land use land cover mapping. Scientific Data, 9, 251. https://doi.org/10.1038/s41597-022-01307-4
Burley, T. M. (1961). Land use and land utilization? The Professional Geographer, 13(6), 18–20. https://doi.org/10.1111/j.0033-0124.1961.136_18.x
Castelluccio, M., Poggi, G., Sansone, C., & Verdoliva, L. (2015). Land use classification in remote sensing images by convolutional neural networks. [Preprint]. ArXiv. https://doi.org/10.48550/arXiv.1508.00092
Cerbaro, M., Morse, S., Murphy, R., Lynch, J., & Griffiths, G. (2020). Information from earth observation for the management of sustainable land use and land cover in Brazil: An analysis of user needs. Sustainability, 12(2), 489. https://doi.org/10.3390/su12020489
Chuvieco, E., & Huete, A. (2009). Fundamentals of satellite remote sensing. CRC Press. https://doi.org/10.1201/b18954
Clark, D. (1982). Urban geography: An introductory guide. Croom Helm.
Cole, B., Smith, G., de la Barreda-Bautista, B., Hamer, A., Payne, M., Codd, T., Johnson, S. C. M., Chan, L. Y., & Balzter, H. (2022). Dynamic landscapes in the UK driven by pressures from energy production and forestry: Results of the CORINE land cover map 2018. Land, 11(2), 192. https://doi.org/10.3390/land11020192
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. https://doi.org/10.1007/BF00994018
Costa, F. J. d., & Urbano, M. R. (2026). Efficient specialization of foundation vision models for urban land cover classification. Artificial Intelligence in Geosciences, 7(2), 100225.
Das, B., Khan, F., & Pir, M. (2023). Impact of urban sprawl on change of environment and consequences. Environmental Science and Pollution Research, 30(49), 106894–106897. https://doi.org/10.1007/s11356-023-29192-3
Deren, L., Zhang, L., & Xia, G.-S. (2014). Automatic analysis and mining of remote sensing big data. Acta Geodaetica et Cartographica Sinica, 43(12), 1211. https://doi.org/10.13485/j.cnki.11-2089.2014.0187
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In J. Burstein, C. Doran, & T. Solorio (Eds.), Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019 [Proceedings]. Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1423
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth 16×16 words: Transformers for image recognition at scale. [Preprint]. ArXiv. https://doi.org/10.48550/arXiv.2010.11929
Esch, T., Heldens, W., & Hirner, A. (2018). The global urban footprint. In Q. Weng, D. Quattrochi, & P. E. Gamba (Eds.), Urban remote sensing (2nd ed., pp. 3–14). CRC Press, Taylor & Francis Group. https://doi.org/10.1201/9781138586642
Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189–1232. https://doi.org/10.1214/aos/1013203451
Gonçalves, D. N., Junior, J. M., Carrilho, A. C., Acosta, P. R., Ramos, A. P. M., Gomes, F. D. G., Osco, L. P., da Rosa Oliveira, M., Martins, J. A. C., Júnior, G. A. D., de Araújo, M. S., Li, J., Roque, F., de Faria Peres, L., Gonçalves, W. N., & Libonati, R. (2023). Transformers for mapping burned areas in Brazilian Pantanal and Amazon with PlanetScope imagery. International Journal of Applied Earth Observation and Geoinformation, 116, 103151. https://doi.org/10.1016/j.jag.2022.103151
Google LLC. (2024). Google Maps Static API. https://developers.google.com/maps/documentation/maps-static
Gorelick, N., Hancher, M., Dixon, M., Ilyushchenko, S., Thau, D., & Moore, R. (2017). Google Earth Engine: Planetary-scale geospatial analysis for everyone. Remote Sensing of Environment, 202, 18–27.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021). LoRA: Low-rank adaptation of large language models. [Preprint]. ArXiv, 1–26. https://doi.org/10.48550/arXiv.2106.09685
Instituto Brasileiro de Geografia e Estatística. (2022). Londrina: População. https://cidades.ibge.gov.br/brasil/pr/londrina/panorama
Instituto Brasileiro de Geografia e Estatística. (2024). Em 2022, PIB cresce em 24 unidades da federação. https://agenciadenoticias.ibge.gov.br/agencia-noticias/2012-agencia-de-noticias/noticias/41893-em-2022-pib-cresce-em-24-unidades-da-federacao
Intituto Brasileiro de Geografia e Estatística. (2023). Censo 2022 indica que o Brasil totaliza 203 milhões de habitantes. https://educa.ibge.gov.br/jovens/conheca-o-brasil/populacao/22005-censo-2022-oretrato-atualizado-do-brasil.html
Jensen, J. R., & Cowen, D. C. (2011). Remote sensing of urban/suburban infrastructure and socio-economic attributes. In M. Dodge, R. Kitchin, & C. Perkins (Eds.), The map reader (pp. 153–163). John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470979587.ch22
Jochem, W. C., Leasure, D. R., Pannell, O., Chamberlain, H. R., Jones, P., & Tatem, A. J. (2021). Classifying settlement types from multi-scale spatial patterns of building footprints. Environment and Planning B: Urban Analytics and City Science, 48(5), 1161–1179. https://doi.org/10.1177/2399808320921208
Lall, S. V., Lebrand, M. S. M., & Soppelsa, M. E. (2021). The evolution of city form: Evidence from satellite data (Policy Research Working Paper No. 9618). The World Bank. https://doi.org/10.1596/1813-9450-9618
Liu, F., Chen, D., Guan, Z., Zhou, X., Zhu, J., Ye, Q., Fu, L., & Zhou, J. (2024). RemoteCLIP: A vision language foundation model for remote sensing. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–16. https://doi.org/10.1109/TGRS.2024.3390838
Lu, H., Shang, Z., Ruan, Y., & Jiang, L. (2023). Study on urban expansion and population density changes based on the inverse S-shaped function. Sustainability, 15(13), 10464. https://doi.org/10.3390/su151310464
Lu, X., Wang, B., Zheng, X., & Li, X. (2018). Exploring models and data for remote sensing image caption generation. IEEE Transactions on Geoscience and Remote Sensing, 56(4), 2183–2195. https://doi.org/10.1109/TGRS.2017.2776321
Penatti, O. A. B., Nogueira, K., & dos Santos, J. A. (2015). Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? In Conference on Computer Vision and Pattern Recognition Workshops (CVPR) [Proceedings] (pp. 44–51). Boston, MA, USA: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/CVPRW.2015.7301382
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020). Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63(10), 1872–1897. https://doi.org/10.48550/arXiv.2003.08271
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In International Conference on Machine Learning [Proceedings] (Vol. 139, pp. 8748–8763). https://proceedings.mlr.press/v139/radford21a.html
Rangel, A., Terven, J., Cordova-Esparza, D. M., & Chavez-Urbiola, E. A. (2024). Land cover image classification. [Preprint]. ArXiv. https://doi.org/10.48550/arXiv.2401.09607
Scheibenreif, L., Hanna, J., Mommert, M., & Borth, D. (2022). Self-supervised vision transformers for land cover segmentation and classification. In R. Chellappa. Conference on Computer Vision and Pattern Recognition [Proceedings]. IEEE/CVF Conference on Computer and Pattern Recognition (CVPR) Workshops. New Orleans, Louisiana, 279–288. https://doi.org/10.1109/CVPRW56347.2022.00036
Sokolova, M., & Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information Processing & Management, 45(4), 427–437. https://doi.org/10.1016/j.ipm.2009.03.002
Souza, C. M., Jr., Shimbo, J. Z., Rosa, M. R., Parente, L. L., Alencar, A. A., Rudorff, B. F. T., Hasenack, H., Matsumoto, M., Ferreira, L. G., Souza-Filho, P. W. M., de Oliveira, S. W., Rocha, W. F., Fonseca, A. V., Marques, C. B., Diniz, C. G., Costa, D., Monteiro, D., Rosa, E. R., Eduardo, ... Azevedo, T. (2020). Reconstructing three decades of land use and land cover changes in Brazilian biomes with Landsat archive and Earth Engine. Remote Sensing, 12(17), 2735. https://doi.org/10.3390/rs12172735
Sparovek, G., Barretto, A., Matsumoto, M., & Berndes, G. (2015). Effects of governance on availability of land for agriculture and conservation in Brazil. Environmental Science & Technology, 49.
Srivastava, P., Mukherjee, S., & Gupta, M. (2010). Impact of urbanization on land use/land cover change using remote sensing and GIS: A case study. International Journal of Ecological Economics and Statistics, 18, 106–117. http://ceser.in/ceserp/index.php/ijees/article/view/1894
Storie, C. D., & Henry, C. J. (2018). Deep learning neural networks for land use land cover mapping. In International Geoscience and Remote Sensing Symposium [Proceedings]. IGARSS 2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain, 6305–6308. https://doi.org/10.1109/IGARSS.2018.8518619
Truong, V. T., Phan, D. C., Nasahara, K., & Tadono, T. (2019). How does land use/land cover map accuracy depend on the number of classification classes? SOLA, 15. https://doi.org/10.2151/sola.2019-006
Ulmas, P., & Liiv, I. (2020). Segmentation of satellite imagery using U-Net models for land cover classification. [Preprint]. ArXiv, 1–11. https://doi.org/10.48550/arXiv.2003.02899
Vali, A., Comai, S., & Matteucci, M. (2020). Deep learning for land use and land cover classification based on hyperspectral and multispectral Earth observation data: A review. Remote Sensing, 12(15), 2495. https://doi.org/10.3390/rs12152495
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In U. von Luxburg, I. Guyon, & S. Bengio, International Conference on Neural Information Processing Systems [Proceedings]. 31st International Conference on Neural Information Processing Systems. Curran Associates Inc., Long Beach, CA, USA, 6000–6010.
https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
Wang, L., Fang, S., Meng, X., & Li, R. (2022). Building extraction with vision transformer. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–11. https://doi.org/10.1109/TGRS.2022.3186634
Wang, R., Ma, L., He, G., Johnson, B. A., Yan, Z., Chang, M., & Liang, Y. (2024). Transformers for remote sensing: A systematic review and analysis. Sensors, 24(11), 3495. https://doi.org/10.3390/s24113495
Weng, Q., Quattrochi, D., & Gamba, P. E. (Eds.). (2018). Urban remote sensing (2nd ed.). CRC Press, Taylor & Francis Group. https://doi.org/10.1201/9781315166612
Zhang, L., Zhang, L., & Du, B. (2016). Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2), 22–40. https://doi.org/10.1109/MGRS.2016.2540798
Zhang, P., Ke, Y., Zhang, Z., Wang, M., Li, P., & Zhang, S. (2018). Urban land use and land cover classification using novel deep learning models based on high spatial resolution satellite imagery. Sensors, 18(11), 3717. https://doi.org/10.3390/s18113717
Zhao, S., Tu, K., Ye, S., Tang, H., Hu, Y., & Xie, C. (2023). Land use and land cover classification meets deep learning: A review. Sensors, 23(21), 8966. https://doi.org/10.3390/s23218966
Zhu, X., & Liang, S. (2020). Urbanization: Monitoring and impact assessment. In S. Liang & J. Wang (Eds.), Advanced remote sensing (2nd ed., pp. 833–870). Academic Press.
Zhu, X. X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F., & Fraundorfer, F. (2017). Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4), 8–36. https://doi.org/10.1109/MGRS.2017.2762307
Zikopoulos, P., Eaton, C., & IBM. (2011). Understanding big data: Analytics for enterprise class Hadoop and streaming data (1st ed.). McGraw-Hill Osborne Media.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Felinto Junior da Costa, Rodrigo Rossetto Pescim, Mariana Ragassi Urbano
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The Copyright Declaration for articles published in this journal is the author's right. Since manuscripts are published in an open access Journal, they are free to use, with their own attributions, in educational and non-commercial applications. The Journal has the right to make, in the original document, changes regarding linguistic norms, orthography, and grammar, with the purpose of ensuring the standard norms of the language and the credibility of the Journal. It will, however, respect the writing style of the authors. When necessary, conceptual changes, corrections, or suggestions will be forwarded to the authors. In such cases, the manuscript shall be subjected to a new evaluation after revision. Responsibility for the opinions expressed in the manuscripts lies entirely with the authors.
This journal is licensed with a license Creative Commons Attribution-NonCommercial 4.0 International.
