Image-based 3D digitizing for plant architecture analysis and phenotyping
3D reconstruction for a basil specimen
Workshop on Industry Applications (WGARI) in SIBGRAPI 2012 (XXV Conference on Graphics, Patterns and Images)
August, 2012
Thiago T. Santos and Alberto A. de Oliveira
Functional-structural plant modeling and plant phenotyping require the measurement of geometric features in specimens. This data acquisition is called plant digitizing. Actually, these measurements are performed manually, in invasive or even destructive ways, or using expensive laser scanning equipment. Computer vision based 3D reconstruction is an accurate and low cost alternative for the digitizing of plants not presenting a dense canopy. Sparse canopies are found in several important annual plants in agriculture as soybean and maize, at least in their early stages of development.
This paper shows as the state of the art in structure from motion and multiple view stereo is able to produce accurate 3D models for specimens presenting sparse canopies. Three-dimensional triangular meshes are computed from a set of non-calibrated images, modeling a basil and an Ixora specimens and accurately representing their leaves and branches.
[pt-BR] Digitalização 3D de plantas por imagem para análise de arquitetura e fenótipo
Série: Embrapa Informática Agropecuária. Documentos, 119
ISSN: 16749274
Edição: 2011
Thiago T. Santos
A digitalização de um objeto pode ser entendida como a construção de um modelo composto por um conjunto discreto de amostras, permitindo o armazenamento e o processamento desse objeto por um computador. O presente trabalho trata da digitalização das características macroscópicas de plantas, sua estrutura e geometria. Diversas análises científicas, entre elas os estudos de correlação fenótipo–genótipo em larga escala, demandam que um grande número de espécimes vegetais seja digitalizado rapidamente. Trata-se portanto de um desafio, já que as técnicas atuais de digitalização de plantas envolvem procedimentos manuais lentos e trabalhosos.
Este trabalho apresenta algumas alternativas para digitalização de plantas, encontradas na literatura recente, que empregam diferentes técnicas de imageamento para obter, de forma automática ou semiautomática, modelos tridimensionais de plantas. Também são apresentadas técnicas de construção de modelos 3D utilizadas em outros domínios que podem ser a base para novos avanços em digitalização de plantas.
Sistemas de digitalização de plantas por imagem, empregando modernas técnicas de reconstrução por visão computacional, podem se tornar uma alternativa eficiente, barata, não-invasiva e não-destrutiva para a produção de um grande volume de dados para análise de fenótipos, modelagem e simulação do desenvolvimento de cultivares. Este trabalho busca apontar alternativas para a construção de tais sistemas.
Automatic camera control in virtual environments augmented using multiple sparse videos
Novel view renderings from the PETS 2006 scenes
Computers & Graphics
Volume 35, Issue 2, April 2011, Pages 412-421
Virtual Reality in Brazil; Visual Computing in Biology and Medicine; Semantic 3D media and content; Cultural Heritage
Jeferson R. Silva, Thiago T. Santos and Carlos H. Morimoto
Automated virtual camera control has been widely used in animation and interactive virtual environments. We have developed a multiple sparse camera based free view video system prototype that allows users to control the position and orientation of a virtual camera, enabling the observation of a real scene in three dimensions (3D) from any desired viewpoint. Automatic camera control can be activated to follow selected objects by the user. Our method combines a simple geometric model of the scene composed of planes (virtual environment), augmented with visual information from the cameras and pre-computed tracking information of moving targets to generate novel perspective corrected 3D views of the virtual camera and moving objects. To achieve real-time rendering performance, view-dependent textured mapped billboards are used to render the moving objects at their correct locations and foreground masks are used to remove the moving objects from the projected video streams. The current prototype runs on a PC with a common graphics card and can generate virtual 2D views from three cameras of resolution 768×576 with several moving objects at about 11 fps.
Multiple camera people detection and tracking using support integration
Multi-view people detection
Pattern Recognition Letters
Volume 32, Issue 1, 1 January 2011, Pages 47-55
Image Processing, Computer Vision and Pattern Recognition in Latin America
Thiago T. Santos and Carlos H. Morimoto
This paper proposes a method to locate and track people by combining evidence from multiple cameras using the homography constraint. The proposed method use foreground pixels from simple background subtraction to compute evidence of the location of people on a reference ground plane. The algorithm computes the amount of support that basically corresponds to the “foreground mass” above each pixel. Therefore, pixels that correspond to ground points have more support. The support is normalized to compensate for perspective effects and accumulated on the reference plane for all camera views. The detection of people on the reference plane becomes a search for regions of local maxima in the accumulator. Many false positives are filtered by checking the visibility consistency of the detected candidates against all camera views. The remaining candidates are tracked using Kalman filters and appearance models. Experimental results using challenging data from PETS’06 show good performance of the method in the presence of severe occlusion. Ground truth data also confirms the robustness of the method.
People Detection under Occlusion in Multiple Camera Views
Five subjects found, marked on the floorplan
XXI Brazilian Symposium on Computer Graphics and Image Processing, 2008. SIBGRAPI ’08
Thiago T. Santos and Carlos H. Morimoto
This paper proposes a method to locate people on a reference plane using multiple cameras. Previous works rely on people trajectories and color models to solve occlusion.This new approach solves people detection under occlusion by accumulating evidence from multiple views instantaneously and does not rely on previous segmentation of individuals in foreground data or any tracking information.First, foreground data from one view, segmented using background subtraction, is projected onto the ground plane or reference image. The projected foreground of a second view overlaps the first projected foreground only on the points where the foreground intersects the ground plane.Thus, by accumulating the evidence from multiple views,people can be located by detecting local maxima on the accumulated reference image. Experimental results using publicly available data from PETSpsila06 [9] show that the method robustly locates people in very challenging situations with occlusion in most of the views. The locations on the ground plane can further be used for segmentation and tracking on each camera view under severe occlusion.
