2019 I/ITSEC

Fully Automated Photogrammetric Data Segmentation and Object Information Extraction Approach for Creating Simulation Terrain (Room 320GH)

04 Dec 19
3:00 PM - 3:30 PM

Tracks: Full Schedule, Wednesday Schedule

Speaker(s): Ryan McAlinden, University of Southern California Institute for Creative Technologies; Meida Chen, University of Southern California Institute for Creative Technologies; Andrew Feng, University of Southern California Institute for Creative Technologies; Lucio Soibelman, University of Southern California Institute for Creative Technologies; Kyle McCullough, University of Southern California Institute for Creative Technologies; Pratusha Bhuvana-Prasad, University of Southern California Institute for Creative Technologies

Our previous works have demonstrated that visually realistic 3D meshes can be automatically reconstructed with low-cost, off-the-shelf unmanned aerial systems (UAS) equipped with capable cameras, and efficient photogrammetric software techniques (McAlinden et al. 2015; Spicer et al. 2016).  However, such generated 3D meshes do not contain semantic information/features of objects (i.e., man-made objects, vegetation, ground, object materials, etc.) and cannot allow sophisticated user-level and system-level interaction. Thus, being able to segment and extract object information from the generated meshes are essential tasks in creating realistic virtual environments for training and simulations.  The objective of this research is to design and develop a fully automated photogrammetric data segmentation and object information extraction framework.  The designed framework utilizes concepts from the areas of computer vision and deep learning. Photogrammetric data are first segmented into different categories (i.e., man-made objects, vegetation, and ground) using deep learning algorithms.  Following that, object information such as individual tree locations and related features and ground materials are extracted with unsupervised and supervised machine learning techniques.  In order to validate the proposed framework, the segmented data and extracted features were used to create virtual environments in the authors’ previously designed simulation tool – the Aerial Terrain Line-of-sight Analysis System (ATLAS).  The results showed that 3D mesh trees can be replaced with geo-typical 3D tree models using the extracted individual tree locations and that the extracted tree features (i.e., color, width, height) are valuable for selecting the appropriate tree species and enhance the visual quality.  Furthermore, results showed that the identified ground material information can be taken into the consideration for path finding.  The shortest path can be computed not only considering the physical distance but also the off-road vehicle performance on different ground surfaces.  An overview of the designed data segmentation workflow is available online (USCICT, 2018).