Completion of the PAgFRUIT project

Download Pdf

The PAgFRUIT research project on Precision Fructiculture has concluded on 30/09/2022. Here below you can find the summary of the main achieved advances and results, together with a list of Open Access publications.

Fruit production in Spain faces a key challenge contemplated in the 2017-2020 Science, Technology and Innovation Research Plan: "Improving the competitiveness and environmental, economic and social sustainability of agricultural production, through the introduction of processes and technologies that increase efficiency and sustainable intensification, including the prevention, protection and control of pests and diseases". This challenge is part of the main challenge 2 on "Bioeconomy: sustainability of primary production systems", with relevant implications in challenge 5 "Climate change and use of natural resources and raw materials", which is where this research project was located. In these challenges, the PAgFRUIT project has contributed to advance in the applicability of Precision Agriculture technologies, such as photonic sensors (LiDAR, RGB-D cameras, intensive photogrammetry, multispectral images from different is types of platforms), to various aspects of fruit growing systems, and particularly in intensive and super-intensive production systems. In summary, the main achievements have been: i) creation of a protocol and a code for the extraction of geometric and structural information from the foliar canopy of intensive and super-intensive fruit tree plantations from 3D point clouds generated by LiDAR sensors, and images acquired from UAV using Structure-from-Motion photogrammetry; ii) characterization of fruit tree canopy through a structural index that correlates with canopy development according to different management strategies (mainly, different nutritional treatments); iii) correlation between vegetation indices calculated from multispectral images from different platforms (Sentinel-2, PlanetScope and drone) with the geometric and structural parameters of the orchards, to serve as a "proxy" in monitoring vigor and development of the foliar architecture of the trees; iv) development of advanced methods based on photonic sensors for the detection, georeferencing and discrimination of fruits, as well as for the measurement of diameter, using artificial intelligence techniques; v) creation of maps of geometric and structural variables of fruit trees based on geostatistical methods, and zoning through cluster analysis, all compared with vigor indices derived from multispectral images; vi) review of the dose decision support system (DOSA3D) for the application of phytosanitary products by calculating zonal doses and validation in vineyard plots with different intra-plot variability (characterized by LiDAR), and comparison of zonal applications using conventional machinery and by dron; vii) proposal of an advanced sampling scheme that, making use of vegetative parameters of the leaf canopy supplied by LiDAR sensors, and in combination with electronic detection of fruits, allows estimating the harvest (fruit load) with greater efficiency than conventional methods. The results obtained consolidate photon-based sensors as a key technology in canopy phenotyping and tree architecture, opening the possibility of implementing new services available to fruit growers for a more optimized management of plantations.

MAIN RESULTS PUBLISHED IN OPEN ACCESS

 

“PEER-REVIEW” ARTILES IN INDEXED JOURNALS

1

Gené-Mola, J.; Gregorio, E., Guevara, J.; Auat, F.; Sanz-Cortiella, R.; Escolà, A.; Llorens, J.; Morros, J.R.; Ruiz-Hidalgo, J.; Vilaplana, V.; Rosell-Polo, J.R., 2019. Fruit detection in an apple orchard using a mobile terrestrial laser scanner. Biosystems Engineering 187, 171-184. POSTPRINT  OPEN ACCESS http://hdl.handle.net/10459.1/66745

2.

Gené-Mola, J.; Gregorio, E.; Auat, F.; Guevara, J.; Llorens, J.; Sanz-Cortiella, R.; Escolà, A.; Rosell-Polo, J.R., 2020-4. Fruit detection, yield prediction and canopy geometric characterization using LiDAR with forced air-flow. Computers and Electronics in Agriculture 168, 105121. POSTPRINT OPEN ACCESS http://hdl.handle.net/10459.1/68782

3

Gené-Mola, J.; Sanz-Cortiella, R., Rosell-Polo, J.R., Morros, J.R, Ruiz-Hidalgo, J., Vilaplana, V., Gregorio, E., 2020-5. Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry. Computers and Electronics in Agriculture 169, 105165. POSTPRINT OPEN ACCESS http://hdl.handle.net/10459.1/67802

4

Guevara, J., Auat, F., Gené-Mola, J., Rosell-Polo, J.R., Gregorio, E., 2020. Analyzing and overcoming the effects of GNSS error on LiDAR based orchard parameters estimation. Computers and Electronics in Agriculture 170, 10525529. PREPRINT OPEN ACCESS http://hdl.handle.net/10459.1/68089

5

Gené-Mola, J., Llorens, J., Rosell-Polo, J.R., Gregorio, E., Arnó, J., Solanelles, F., Martínez-Casasnovas, J.A., Escolà, A., 2020-1. Assessing the Performance of RGB-D Sensors for 3D Fruit Crop Canopy Characterization under Different Operating and Lighting Conditions. Sensors 20(24), 7072; ARTÍCULO OPEN ACCESS https://doi.org/10.3390/s20247072

6

Gené-Mola, J., Sanz-Cortiella, R., Rosell-Polo, J.R., Escolà, A., Gregorio, E., 2021-1. In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions. Computers and Electronics in Agriculture 188, 106343. ARTÍCULO OPEN ACCESS https://doi.org/10.1016/j.compag.2021.106343

7

Lavaquiol, B., Sanz, R., Llorens, J. Arnó, J., Escolà, A., 2021. A photogrammetry-based methodology to obtain accurate digital ground-truth of leafless fruit trees. Computers and Electronics in Agriculture 191, 06553. ARTÍCULO OPEN ACCESS https://doi.org/10.1016/j.compag.2021.106553

8

Guevara, D.J., Gené-Mola, J., Gregorio, E., Auat Cheein, F., 2022. 3D Spectral Graph Wavelet Point Signatures in Pre-Processing Stage for Mobile Laser Scanning Point Cloud Registration in Unstructured Orchard EnvironmentsIEEE Sensors Journal. POSTPRINT  OPEN ACCESS  http://hdl.handle.net/10459.1/72508 

9

Guevara, D.J., Gené-Mola, J., Gregorio, E., Torres-Torriti, M., Reina, G., Auat Cheein, F., 2021-1. Comparison of 3D scan matching techniques for autonomous robot navigation in urban and agricultural environments. Journal of Applied Remote Sensing 16(2), 024508. POSTPRINT  OPEN ACCESS   http://hdl.handle.net/10459.1/71527

10

Pérez, G., Escolà, A, Rosell-Polo, J.R., Coma, J., Arasanz, R., Marrero, B., Cabeza, L.F., Gregorio, E., 2021. 3D characterization of a Boston Ivy double-skin green building facade using a LiDAR system. Building and Environment 206 (2021), 108320. ARTÍCULO  OPEN ACCESS  https://doi.org/10.1016/j.buildenv.2021.108320

11

Martínez-Casasnovas, J.A., Sandonís-Pozo, L., Escolà, A., Arnó, J., Llorens, J., 2022. Delineation of management zones in super-intensive almond orchards based on vegetation indices from UAV images validated by LiDAR-derived canopy parameters. Agronomy 12(1), 102. ARTÍCULO  OPEN ACCESS  https://doi.org/10.3390/agronomy12010102

12

Sandonís-Pozo, L., Llorens, J., Escolà, A., Arnó, J., Pascual, M., Martínez-Casasnovas, J.A., 2022-1. Satellite multispectral indices to estimate canopy parameters and within‑field management zones in super‑intensive almond orchards. Precision Agriculture, 2022. ARTÍCULO  OPEN ACCESS  https://doi.org/10.1007/s11119-022-09956-6

13

Miranda, J.C., Gené-Mola, J., Arnó, J., Gregorio, E., 2022-1. AKFruitData: a dual software application for Azure Kinect cameras to acquire and extract informative data in yield tests performed in fruit orchard environments.  SoftwareX 20, 101231.  ARTÍCULO  OPEN ACCESS https://doi.org/10.1016/j.softx.2022.101231

 

DATA ARTICLES

14

Gené-Mola, J., Sanz-Cortiella, R., Rosell-Polo, J.R., Morros, J.R., Ruiz-Hidalgo, J., Vilaplana, V., Gregorio, E., 2020-6. Fuji-SfM dataset: a collection of annotated images and point clouds for Fuji apple detection and location using structure-from-motion photogrammetry. Data in Brief 29 (2020), 105591. ARTÍCULO DE DATOS OPEN ACCESS http://doi.org/10.1016/j.dib.2020.105591

15

Gené-Mola, J.; Gregorio, E.; Auat, F.; Guevara, J.; Llorens, J.; Sanz-Cortiella, R.; Escolà, A.; Rosell-Polo, J.R., 2020-4. LFuji-air dataset: Annotated 3D LiDAR point clouds of Fuji apple trees for fruit detection scanned under different forced air flow conditions. Data in Brief 29 (2020), 105248.  ARTÍCULO DE DATOS OPEN ACCESS  https://doi.org/10.1016/j.dib.2020.105248

16

Gené-Mola, J., Sanz-Cortiella, R., Rosell-Polo, J.R., Escolà, A., Gregorio, E., 2021-2. PFuji-Size dataset: A collection of images and photogrammetry-derived 3D point clouds with ground truth annotations for Fuji apple detection and size estimation in field conditions. Data in Brief 39, 107629.  ARTÍCULO DE DATOS OPEN ACCESS https://doi.org/10.1016/j.dib.2021.107629

 

ARTICLES IN DISSEMINATION JOURNALS

17

Gené-Mola, J., Gregorio, E., Rosell-Polo, J.R., 2020-0. Cómo la inteligencia artificial nos ayuda a contar manzanas. The Conversation (Ciencia y Tecnología), 27/01/2020. ARTÍCULO DIVULGATIVO OPEN ACCESS https://theconversation.com/como-la-inteligencia-artificial-nos-ayuda-a-contar-manzanas-130571

 

BOOK CHAPTERS

18

Martínez-Casasnovas, J.A., Arnó, J., Escolà, A., 2022. Sensores de conductividad eléctrica aparente para el análisis de la variabilidad del suelo en Agricultura de Precisión.  En: A. Namesny, C. Conesa, L.M. Olmos, P. Papasseit (Eds), "Tecnología Hortícola Mediterránea. Evolución y futuro: viveros, frutales, hortalizas y ornamentales". Biblioteca de Horticultura, SPE3 S.L., Valencia, España. 1075 pp. ISBN: 978-84-16909-46-9. CAPÍTULO LIBRO OPEN ACCESS https://issuu.com/horticulturaposcosecha/docs/tecnologia_horticola_mediterranea, pag 765-786.

 

PROTOCOLS AND CODES FOR DATA PROCESSING AND WEB APPLICATIONS

19

Protocolo de Clasificación de la nube de puntos generada por el equipo Viametris disponible en el apartado 3 de este trabajo: http://hdl.handle.net/10459.1/70512. Y código R-RStudio para la extracción de parámetros vegetativos disponible en el apartado de anejos de este trabajo: http://hdl.handle.net/10459.1/70369

20

Revisión de la aplicación DOSA3D – Dosis Zonal: http://www.dosa3d.cat/es/introduction

21

Miranda JC, Gené-Mola J, Arnó J, Gregorio E, 2022-2. Herramienta GUI basada en Python para extraer imágenes de archivos de video producidos con cámaras Kinect Azure. AK_FRAEX - Azure Kinect Frame Extractor. https://pypi.org/project/ak-frame-extractor/

22

Miranda JC, Gené-Mola J, Arnó J, Gregorio E, 2022-3. A simple GUI recorder based on Python to manage Zaure Kinect camera divices in a standalone mode. https://pypi.org/project/ak-sm-recorder/

23

Aplicación informática AKFruitData: A dual software application for Azure Kinect cameras to acquire and extract informative data in yield tests performed in fruit orchard environments. https://github.com/GRAP-UdL-AT/SOFTX_SOFTX-D-22-00152

24

Gené-Mola J, Sanz-Cortiella R, Rosell-Polo JR, Morros JR, Ruiz-Hidalgo J, Vilaplana V, Gregorio E., 2020. Proyecto Matlab para proyectar detecciones de imágenes en nubes de puntos 3D generadas mediante estructura a partir del movimiento https://github.com/GRAP-UdL-AT/SfM_3D_fruit_detection

25

Gené-Mola J, Sanz-Cortiella R, Rosell-Polo JR, Escolà A, Gregorio E., 2021. Proyecto Matlab para la estimación del tamaño de manzanas en nubes de puntos 3D https://github.com/GRAP-UdL-AT/apple_size_estimation_in_3D_point_clouds

26

Felip Pomés M, Net Barnes F, Gené Mola J, 2022. Proyecto Python para la detección y seguimiento de frutas utilizando YOLOv5 y ByteTrack. El proyecto fue construido para ser probado en conjunto con el robot AMIGA para fines de conteo de frutas en el campo. Implementado en el Farm@thon organizado por Lleida Drone y Farm-ng, que obtuvo el primer premio del concurso Farm@thon 2022 https://github.com/GRAP-UdL-AT/AMIGA_fruit_counting

27

Gené-Mola J, Llorens J, Rosell-Polo JR, Gregorio E, Arnó J, Solanelles F, Martínez-Casasnovas JA, Escolà A., 2020. Proyecto Matlab para evaluar el rendimiento del sensor RGB-D mediante el análisis de los datos RGB-D adquiridos en diferentes condiciones de plantaciones frutales https://github.com/GRAP-UdL-AT/RGBD_sensors_evaluation_in_Orchards

28

Gené-Mola J, Gregorio E, Auat F, Guevara J, Llorens J, Sanz-Cortiella R, Escolà A, Rosell-Polo JR, 2020. Proyecto Matlab para la detección de frutas en nubes de puntos 3D adquiridas con el sensor LiDAR Velodyne VLP-16 https://github.com/GRAP-UdL-AT/fruit_detection_in_LiDAR_pointClouds

29

Gené-Mola J, Gregorio E, Sanz-Cortiella R, Escolà A, Llorens J, Rosell-Polo JR., 2019. Proyecto Matlab para generar nubes de puntos 3D a partir de datos adquiridos con un escáner láser terrestre móvil (MTLS) compuesto por un sensor LiDAR Velodyne VLP-16 y GNSS GPS1200+ https://github.com/GRAP-UdL-AT/MTLS_point_cloud_generation

 

DATA

30

Gené-Mola, J., Llorens, J., Rosell-Polo, J.R., Gregorio, E., Arnó, J., Solanelles, F., Martínez-Casasnovas, J.A., Escolà, A., 2020-2. KEvOr dataset. Zenodo, 4286460. DATOS OPEN ACCESS https://doi.org/10.5281/zenodo.4286460

31

Guevara, D.J., Gené-Mola, J., Gregorio, E., Torres-Torriti, M., Reina, G., Auat Cheein, F. 2021-2. AgLiMatch dataset [Data set]. Zenodo. DATOS OPEN ACCESS  https://doi.org/10.5281/zenodo.4415385

 

ARTICLES IN CONGRESSES

32

Llorens, J.; Cabrera Pérez; C., Escolà, A.; Arnó, J., 2019. R software code to process and extract information from 3D Lidar point clouds. En: Poster Proceedings of the 12th European Conference on Precision Agriculture, July 8-11, Montpellier, France. pp. 114-115. e-book publication. SupAgro Montpellier. ISBN 978-2-900792-49-0. http://hdl.handle.net/10459.1/84290

33

Gené-Mola, J.; Sanz-Cortiella, R.; Rosell-Polo, J.R.; Morros, J.R.; Ruiz-Hidalgo, J.; Vilaplana, V.; Gregorio, E., 2020-7. Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry. 7th Annual Catalan Meeting on Computer Vision. September 22, 2020, Universitat Autònoma de Barcelona, (http://acmcv.cat/).  http://hdl.handle.net/10459.1/84029

34

Llorens, J., Alsina, A., Arnó, J., Martínez-Casasnovas, J.A., Escolà, A., 2021-1. Multi-beam LiDAR-derived data analysis for optimal canopy 3D monitoring in super-intensive almond (Prunus dulcis) orchards. In: Stafford, J.V. (ed.), Precision Agriculture’21. Wageningen Academic Publishers, Amsterdam (The Netherlands), pp 395-401. DOI: 10.3920/978-90-8686-916-9. PREPRINT OPEN ACCESS:   http://hdl.handle.net/10459.1/84065

35

Llorens, J., Escolà, A., Casañas, E., Rosell-Polo, J.R., Arnó, J., Martínez-Casasnovas, J.A., 2021-2. Estimation of geometric and structural parameters in a super-intensive almond (Prunus dulcis) orchard from multispectral vegetation indices derived from a UAV image. In: Stafford, J.V. (ed.), Precision Agriculture’21. Wageningen Academic Publishers, Amsterdam (The Netherlands), pp 129-135. DOI: 10.3920/978-90-8686-916-9.    PREPRINT OPEN ACCESS:  http://hdl.handle.net/10459.1/84060

36

Martínez-Casasnovas, J.A., Llorens, J., Sandonís-Pozo, L., Escolà, A., Arnó, J., 2021. NDVI from satellite images to estimate LiDAR-derived geometric and structural parameters in super-intensive almond orchards. In: Stafford, J.V. (ed.), Precision Agriculture’21. Wageningen Academic Publishers, Amsterdam (The Netherlands), pp 567-572. DOI: 10.3920/978-90-8686-916-9. PREPRINT OPEN ACCESS   http://hdl.handle.net/10459.1/84063

37

Lavaquiol, B.; Llorens, J.; Sanz, R.; Gené, J., Arnó, J.; Gregorio, E.; Escolà, A., 2021. Metodología para el análisis de los errores y validación de nubes de puntos 3D obtenidas en campo para la caracterización de la arquitectura de árboles frutales. XI Congreso Ibérico de Agroingeniería (virtual). http://hdl.handle.net/10459.1/84291

38

Sandonís-Pozo, L., Plata-Moreno, J.M., Llorens, J., Escolà, A., Pascual, M. Martínez-Casasnovas, J.A., 2022-2. PlanetScope Vegetation Indices to Estimate UAV and LiDAR-derived Canopy Parameters in a Super-Intensive Almond Orchard. 14th International Symposium FRUTIC 2022, June 29 – July 1, 2022, Valencia. http://hdl.handle.net/10459.1/84009 

39

Gené-Mola, J., Sanz-Cortiella, R., Rosell-Polo, J.R., Escolà, A., Gregorio, E., 2022-1. Apple size estimation using photogrammetry-derived 3D point clouds. Anual Catalan Meeting on Computer Vision 2022, Universitat Autònoma de Barcelona, (http://acmcv.cat/). http://hdl.handle.net/10459.1/84013

40

Gené Mola, J.; Gregorio, E.; Sanz Cortiella, R.; Escolà, A.; Rosell Polo, J.R., 2022-2.  Fruit detection and sizing using photonic sensors and artificial intelligence. International Conference on AI Applications in Agriculture. International Center for Biosaline Agriculture (ICBA) and Universitat de Barcelona (UB). July 19-20, 2022. Barcelona.   http://hdl.handle.net/10459.1/84030

41

Martínez-Casasnovas, J.A., 2021. Aplicaciones de la teledetección en la caracterización de frutales y en la fertilización de cultivos extensivos. III Jornadas Científico-Técnicas de Teledetección y Agricultura de Precisión (Lleida, España). CONFERENCIA/WORKSHOP. http://hdl.handle.net/10459.1/84138

42

Sandonís-Pozo, L., Llorens, J., Escolà, A., Arnó, J., Pascual, M., Martínez-Casasnovas, J.A., 2022-3. Assessment of different N treatments in Hedgerow Almond Orchards by means of LiDAR point clouds. XXI International Nitrogen Workshop 2022. Madrid 24-28 Octubre 2022. http://hdl.handle.net/10459.1/84121

43

Sandonís-Pozo, L., Arnó, J., Rufat, J., Villar, J.M., Martínez-Casasnovas, J.A., Pascual, M., 2022-4. Análisis del dosel foliar de setos de olivo mediante LiDAR y su relación con la productividad y atributos de calidad del aceite. VII Jornadas Nacionales del Grupo de Olivicultura de la Sociedad Española de Ciencias Hortícolas (SECH). Logroño 19-20 Octubre 2022. http://hdl.handle.net/10459.1/84118

44

Cabrera-Pérez, C., Llorens, J., Escolà, A., Royo-Esnal, A., Recasens, J., 2022.  Manejo de malas hierbas bajo la línea del viñedo mediante acolchados orgánicos y su efecto sobre el vigor del cultivo. En: G. Santesteban y N. Torres (Eds.), Acta de Horticultura 91, 390-394.  http://hdl.handle.net/10459.1/84189

 

DOCTORAL THESIS, FINAL DEGREE AND MASTER THESIS DISSERTATIONS

45

Gené Mola, J., 2020. Fruit detection and 3D location using optical sensors and computer vision. Directores: E. Gregorio & J.R. Rosell Polo. Tesis Doctoral OPEN ACCESS: http://hdl.handle.net/10803/669110 (Premio Extraordinario de doctorado)

46

Alsina Theas, A., 2020. Determinación de los parámetros óptimos de escaneo del sensor LiDAR Velodyne en plantaciones de almendros súper-intensivos. Máster en Ing. Agronómica, UdL. Tutores: J. Llorens, A. Escolà.  TFM OPEN ACCESS:  http://hdl.handle.net/10459.1/70512  

47

Maestro Balaguer, S., 2020. Puesta en marcha y evaluación experimental de una cámara RGB-D: Microsoft Azure Kinect. Máster en Ing. Industrial, UdL. Tutor: E. Gregorio. TFM OPEN ACCESS: http://hdl.handle.net/10459.1/69627

48

Rosell Tarragó, M., 2020. Design of a 3D photogrammetry acquisition system and data processing workflow automation. TFG Grado Ing. Mecánica, UdL.Tutores: A. Escolà, R. Sanz. TFG OPEN ACCESS: http://hdl.handle.net/10459.1/70339 

49

Raduà Castellví, P., 2020. Detección y caracterización geométrica de frutos utilizando técnicas de fotogrametría. TFG Grado Ing. Agraria y Alimentaria, UdL Tutor: R. Sanz.  TFG OPEN ACCESS: http://hdl.handle.net/10459.1/70380  

50

Rotés Biosca, A., 2021. Prestaciones de una cámara monofocal para la caracterización de árboles frutales desde plataformas aéreas y terrestres. Medida del NDVI y comparación con sensores de relfectancia. Grado de Ing. Agraria y Alimentaria. UdL. Tutores A. Escolà, J. Arnó. TFG OPEN ACCESS: http://hdl.handle.net/10459.1/72242  

51

Camats, H., 2020. Fenotipado mediante un sensor LiDAR terrestre de una plantación de almendros bajo un ensayo experimental de fertirrigación. Grado de Ing. Agraria y Alimentaria, UdL. Tutores: J. Arnó, J. Llorens. TFG OPEN ACCESS: http://hdl.handle.net/10459.1/70369  

52

Casañas, E., 2020. Anàlisi de paràmetres vegetatius en plantacions intensives d’ametllers mitjançant diferents tècniques de teledetecció. Grado de Ing. Agraria y Alimentaria, UdL Tutor: J.A. Martínez, J. Llorens. TFG OPEN ACCESS:  http://hdl.handle.net/10459.1/84140

 53

Ferrer Ferrer, M., 2021. Fruit size estimation using Multitask Deep Neural Networks. Master Thesis in Computer Vision. Universitat Politècnica de Catalunya. Supervisors: J. Ruiz Hidalgo (UPC) & J. Gené Mola (UdL). TFM: https://unidisc.csuc.cat/oc-shib/index.php/s/8ryaSG2MFClhCIl

 54

Simón, P., 2022. Detecció i seguiment visual de fruites. Trabajo final de grado en Ingeniería de Telecomunicaciones. Universitat Politècnica de Catalunya. Supervisores: R. Morros (UPC) & J. Gené Mola (UdL). TFG: https://unidisc.csuc.cat/index.php/s/sA1InDfxITsxi54

 55

Net Barnés, F., 2022. Fruit detection and tracking in RGB-D videos. Master Thesis in Computer Vision. Universitat Autònoma de Barcelona (UAB). Supervisors: J. Gené Mola (UdL) & R. Morros (UPC). TFM: https://unidisc.csuc.cat/index.php/s/sQ2Xdv8OO3DO6lN

 56

Galve, I., 2022. 3D apple segmentation and measurement in large unstructured point clouds. Master Thesis in Computer Vision. Universitat Autònoma de Barcelona (UAB). Supervisors: J. Ruiz-Hidalgo (UPC), J. Gené Mola (UdL) & V. Vilaplana (UPC). TFM: https://unidisc.csuc.cat/index.php/s/43cWuyyTe6KMa8gN