Kiretu
Public Member Functions | Private Attributes

KinectCloud Class Reference

C++-class which represents a point-cloud of Microsoft's Kinect and allows the reconstruction of a point-cloud applying the steps explained at the documentation. More...

#include <KinectCloud.h>

Public Member Functions

 KinectCloud ()
 Constructor of the KinectCloud-class.
 ~KinectCloud ()
 Destructor of the KinectCloud-class.
void addZValues (int *zValues)
 Adds the z-values of a Kinect depth-image grabbed by FrameGrabber and stores them as a point-cloud (std::vector< std::vector<float> >).
void addRgbValues (int *rgbValues)
 Adds the RGB-values of a Kinect RGB-image grabbed by FrameGrabber and stores them as a point-cloud (std::vector< std::vector<float> >).
void rawToMeter ()
 Converts the added depth-values of the Kinect ( $ Z_\mathrm{raw} \in [0, 2047) $) into meter using the formula

\[ Z_\mathrm{meter} = \frac{1}{Z_\mathrm{raw} \cdot (-0.0030711016) + 3.3309495161}, \]

which was experimentaly determined (http://nicolas.burrus.name/index.php/Research/KinectCalibration).

void depthToCloud (std::vector< std::vector< float > > depthCam)
 Applies the intrinsic camera-parameters to the point-cloud. The parameters should be imported using YMLParser. With the help of these parameters, each point $ (x,y)^T $ of the depth-image with its depth-value $Z$ is projected on its equivalent point $ (X,Y,Z)^T $ in 3D-space regarding its z-value and the depth-camera-parameters. This is done with the following formula:

\begin{align*} \begin{pmatrix} x_\mathsf{d} \\ y_\mathsf{d}\\ 1 \end{pmatrix} &= \begin{pmatrix} & & & 0 \\ & \mathbf{C} & & 0 \\ & & & 0 \end{pmatrix} \begin{pmatrix} X_\mathsf{d} \\ Y_\mathsf{d} \\ Z_\mathsf{d} \\ 1 \end{pmatrix} = \begin{pmatrix} f_x & 0 & c_x & 0 \\ 0 & f_y & c_y & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} \cdot \begin{pmatrix} X_\mathsf{d} \\ Y_\mathsf{d} \\ Z_\mathsf{d} \\ 1 \end{pmatrix} \\ &= \begin{pmatrix} f_x \cdot X_\mathsf{d} + c_x \cdot Z_\mathsf{d} \\ f_y \cdot Y_\mathsf{d} + c_y \cdot Z_\mathsf{d} \\ Z \end{pmatrix} = \begin{pmatrix} f_x \cdot X_\mathsf{d} / Z_\mathsf{d} + c_x \\ f_y \cdot Y_\mathsf{d} / Z_\mathsf{d} + c_y \\ 1 \end{pmatrix} \\ &\Rightarrow\quad \begin{cases} X_\mathsf{d} = (x_\mathsf{d} - c_x) \cdot Z_\mathsf{d} / f_x \\ Y_\mathsf{d} = (y_\mathsf{d} - c_y) \cdot Z_\mathsf{d} / f_y \end{cases}, \end{align*}

where $\mathbf{C}$ contains the intrinsic parameters of the depth-camera. Notice that the z-value doesn't change during this procedure and is already available at the depth-image.

void applyExtrinsics (std::vector< std::vector< float > > rot, std::vector< float > trans)
 Applies the extrinsic camera-parameters to the point-cloud. The parameters should be imported using YMLParser. This is done with a rotation and translation of each point:

\[ \begin{pmatrix} x_\mathrm{rgb} \\ y_\mathrm{rgb} \\ z_\mathrm{rgb} \end{pmatrix} = R \cdot \begin{pmatrix} x_\mathrm{d} \\ y_\mathrm{d} \\ z_\mathrm{d} \end{pmatrix} + T = \begin{pmatrix} r_{11} & r_{12} & r_{13} \\ r_{21} & r_{22} & r_{23} \\ r_{31} & r_{32} & r_{33} \end{pmatrix} \cdot \begin{pmatrix} x_\mathrm{d} \\ y_\mathrm{d} \\ z_\mathrm{d} \end{pmatrix} + \begin{pmatrix} t_1 \\ t_2 \\ t_3 \end{pmatrix}, \]

where $R$ and $T$ are the extrinsic parameters of the Kinect and describe a rotation and translation.

void computeRgbMapping (std::vector< std::vector< float > > rgbCam)
 Computes the mapping between the point-cloud and the equivalent points of the RGB-image by reprojecting the point $ (X_\mathsf{rgb},Y_\mathsf{rgb},Z_\mathsf{rgb}) $ of the point-cloud to its equivalent point $ (x_\mathsf{rgb},y_\mathsf{rgb}) $ in the plane using the intrinsics of the RGB-camera imported by YMLParser. The projection is done with the following formula:

\begin{align*} \begin{pmatrix} x_\mathsf{rgb} \\ y_\mathsf{rgb} \\ 1 \end{pmatrix} &= \begin{pmatrix} & & & 0 \\ & \mathbf{C} & & 0 \\ & & & 0 \end{pmatrix} \begin{pmatrix} X_\mathsf{rgb} \\ Y_\mathsf{rgb} \\ Z_\mathsf{rgb} \\ 1 \end{pmatrix} = \begin{pmatrix} f_x & 0 & c_x & 0 \\ 0 & f_y & c_y & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} \cdot \begin{pmatrix} X_\mathsf{rgb} \\ Y_\mathsf{rgb} \\ Z_\mathsf{rgb} \\ 1 \end{pmatrix} \\ &= \begin{pmatrix} f_x \cdot X_\mathsf{rgb} + c_x \cdot Z_\mathsf{rgb} \\ f_y \cdot Y_\mathsf{rgb} + c_y \cdot Z_\mathsf{rgb} \\ Z \end{pmatrix} = \begin{pmatrix} f_x \cdot X_\mathsf{rgb} / Z + c_x \\ f_y \cdot Y_\mathsf{rgb} / Z + c_y \\ 1 \end{pmatrix} \\ &\Rightarrow \begin{cases} x_\mathsf{rgb} = (f_x \cdot X_\mathsf{rgb} / Z_\mathsf{rgb}) + c_x \\ y_\mathsf{rgb} = (f_y \cdot Y_\mathsf{rgb} / Z_\mathsf{rgb}) + c_y \end{cases}, \end{align*}

where $C$ contains the intrinsic parameters of the RGB-camera.

std::vector< std::vector< float > > getCloud ()
 Get the cloud.
std::vector< std::vector< int > > getCloudRgb ()
 Get the RGB-cloud.
std::vector< std::vector< int > > getCloudRgbMapping ()
 Get the cloud-RGB-mapping.
std::vector< bool > getValidValues ()
 Get a vector which indicates if a point of the point-cloud is valid ( $depth < 2047$).
std::vector< bool > getReconstructionSteps ()
 Get the reconstruction-steps.

Private Attributes

std::vector< std::vector< float > > cloud
std::vector< std::vector< int > > cloudRgb
std::vector< std::vector< int > > cloudRgbMapping
std::vector< bool > validValues
std::vector< bool > reconstructionSteps

Detailed Description

C++-class which represents a point-cloud of Microsoft's Kinect and allows the reconstruction of a point-cloud applying the steps explained at the documentation.

Author:
Daniel Wunderlich (d.wunderlich@stud.uni-heidelberg.de)
Version:
0.8
Date:
2011-01-26
See also:
Reconstruction and KinectCloud

Constructor & Destructor Documentation

KinectCloud::KinectCloud ( )

Constructor of the KinectCloud-class.

KinectCloud::~KinectCloud ( )

Destructor of the KinectCloud-class.

Member Function Documentation

void KinectCloud::addRgbValues ( int *  rgbValues)

Adds the RGB-values of a Kinect RGB-image grabbed by FrameGrabber and stores them as a point-cloud (std::vector< std::vector<float> >).

Parameters:
rgbValuesPointer to a $640 \times 480$-dimensional array of RGB-values.
void KinectCloud::addZValues ( int *  zValues)

Adds the z-values of a Kinect depth-image grabbed by FrameGrabber and stores them as a point-cloud (std::vector< std::vector<float> >).

Parameters:
zValuesPointer to a $640 \times 480$-dimensional array of z-values.
void KinectCloud::applyExtrinsics ( std::vector< std::vector< float > >  rot,
std::vector< float >  trans 
)

Applies the extrinsic camera-parameters to the point-cloud. The parameters should be imported using YMLParser. This is done with a rotation and translation of each point:

\[ \begin{pmatrix} x_\mathrm{rgb} \\ y_\mathrm{rgb} \\ z_\mathrm{rgb} \end{pmatrix} = R \cdot \begin{pmatrix} x_\mathrm{d} \\ y_\mathrm{d} \\ z_\mathrm{d} \end{pmatrix} + T = \begin{pmatrix} r_{11} & r_{12} & r_{13} \\ r_{21} & r_{22} & r_{23} \\ r_{31} & r_{32} & r_{33} \end{pmatrix} \cdot \begin{pmatrix} x_\mathrm{d} \\ y_\mathrm{d} \\ z_\mathrm{d} \end{pmatrix} + \begin{pmatrix} t_1 \\ t_2 \\ t_3 \end{pmatrix}, \]

where $R$ and $T$ are the extrinsic parameters of the Kinect and describe a rotation and translation.

Consider that the z-values should be converted into meters (rawToMeter()) before calling this function for a correct reconstruction.

Parameters:
rotRotation-matrix ( $ 3 \times 3 $) of the Kinect as written in the specific yml-file and imported using YMLParser.
transTranslation-vector (3-dimensional) of the Kinect as written in the specific yml-file and imported using YMLParser.
See also:
Step 3: Apply extrinsics
void KinectCloud::computeRgbMapping ( std::vector< std::vector< float > >  rgbCam)

Computes the mapping between the point-cloud and the equivalent points of the RGB-image by reprojecting the point $ (X_\mathsf{rgb},Y_\mathsf{rgb},Z_\mathsf{rgb}) $ of the point-cloud to its equivalent point $ (x_\mathsf{rgb},y_\mathsf{rgb}) $ in the plane using the intrinsics of the RGB-camera imported by YMLParser. The projection is done with the following formula:

\begin{align*} \begin{pmatrix} x_\mathsf{rgb} \\ y_\mathsf{rgb} \\ 1 \end{pmatrix} &= \begin{pmatrix} & & & 0 \\ & \mathbf{C} & & 0 \\ & & & 0 \end{pmatrix} \begin{pmatrix} X_\mathsf{rgb} \\ Y_\mathsf{rgb} \\ Z_\mathsf{rgb} \\ 1 \end{pmatrix} = \begin{pmatrix} f_x & 0 & c_x & 0 \\ 0 & f_y & c_y & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} \cdot \begin{pmatrix} X_\mathsf{rgb} \\ Y_\mathsf{rgb} \\ Z_\mathsf{rgb} \\ 1 \end{pmatrix} \\ &= \begin{pmatrix} f_x \cdot X_\mathsf{rgb} + c_x \cdot Z_\mathsf{rgb} \\ f_y \cdot Y_\mathsf{rgb} + c_y \cdot Z_\mathsf{rgb} \\ Z \end{pmatrix} = \begin{pmatrix} f_x \cdot X_\mathsf{rgb} / Z + c_x \\ f_y \cdot Y_\mathsf{rgb} / Z + c_y \\ 1 \end{pmatrix} \\ &\Rightarrow \begin{cases} x_\mathsf{rgb} = (f_x \cdot X_\mathsf{rgb} / Z_\mathsf{rgb}) + c_x \\ y_\mathsf{rgb} = (f_y \cdot Y_\mathsf{rgb} / Z_\mathsf{rgb}) + c_y \end{cases}, \end{align*}

where $C$ contains the intrinsic parameters of the RGB-camera.

Consider that the depth-image needs to be projected into the 3D-space using (depthToCloud(std::vector< std::vector<float> > depthCam)) before calling this function for correct reconstruction.

Parameters:
rgbCamIntrisic parameters of the RGB-cam as a $ 3 \times 3 $-matrix as written in the specific yml-file and imported using YMLParser.
See also:
Step 4: RGB mapping
void KinectCloud::depthToCloud ( std::vector< std::vector< float > >  depthCam)

Applies the intrinsic camera-parameters to the point-cloud. The parameters should be imported using YMLParser. With the help of these parameters, each point $ (x,y)^T $ of the depth-image with its depth-value $Z$ is projected on its equivalent point $ (X,Y,Z)^T $ in 3D-space regarding its z-value and the depth-camera-parameters. This is done with the following formula:

\begin{align*} \begin{pmatrix} x_\mathsf{d} \\ y_\mathsf{d}\\ 1 \end{pmatrix} &= \begin{pmatrix} & & & 0 \\ & \mathbf{C} & & 0 \\ & & & 0 \end{pmatrix} \begin{pmatrix} X_\mathsf{d} \\ Y_\mathsf{d} \\ Z_\mathsf{d} \\ 1 \end{pmatrix} = \begin{pmatrix} f_x & 0 & c_x & 0 \\ 0 & f_y & c_y & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} \cdot \begin{pmatrix} X_\mathsf{d} \\ Y_\mathsf{d} \\ Z_\mathsf{d} \\ 1 \end{pmatrix} \\ &= \begin{pmatrix} f_x \cdot X_\mathsf{d} + c_x \cdot Z_\mathsf{d} \\ f_y \cdot Y_\mathsf{d} + c_y \cdot Z_\mathsf{d} \\ Z \end{pmatrix} = \begin{pmatrix} f_x \cdot X_\mathsf{d} / Z_\mathsf{d} + c_x \\ f_y \cdot Y_\mathsf{d} / Z_\mathsf{d} + c_y \\ 1 \end{pmatrix} \\ &\Rightarrow\quad \begin{cases} X_\mathsf{d} = (x_\mathsf{d} - c_x) \cdot Z_\mathsf{d} / f_x \\ Y_\mathsf{d} = (y_\mathsf{d} - c_y) \cdot Z_\mathsf{d} / f_y \end{cases}, \end{align*}

where $\mathbf{C}$ contains the intrinsic parameters of the depth-camera. Notice that the z-value doesn't change during this procedure and is already available at the depth-image.

Consider that the extrinsic parameters should already be applied to the point-cloud (applyExtrinsics(std::vector< std::vector<float> > rot, std::vector<float> trans)) before calling this function.

Parameters:
depthCamIntrisic parameters of the depth-cam as a $ 3 \times 3 $-matrix as written in the specific yml-file and imported using YMLParser.
See also:
Step 2: Depth to cloud
std::vector< std::vector< float > > KinectCloud::getCloud ( )

Get the cloud.

Returns:
Vector of 3-dimensional points.
std::vector< std::vector< int > > KinectCloud::getCloudRgb ( )

Get the RGB-cloud.

Returns:
Vector of 3-dimensional vectors containing the RGB-values of the cloud.
std::vector< std::vector< int > > KinectCloud::getCloudRgbMapping ( )

Get the cloud-RGB-mapping.

Returns:
Vector of 2-dimensional vectors, which contain the x- and y-value of the pixel at the RGB-image of every point at the point-cloud. It is used to find the corresponding color of every point of the point-cloud at the RGB-image.
std::vector< bool > KinectCloud::getReconstructionSteps ( )

Get the reconstruction-steps.

Returns:
Vector boolean values indicating if the reconstruction step was done. The values of a returned vector recon indicate:
std::vector< bool > KinectCloud::getValidValues ( )

Get a vector which indicates if a point of the point-cloud is valid ( $depth < 2047$).

Returns:
Indicates for every of the $640 \times 480$ points if it is valid ( $depth < 2047$).
void KinectCloud::rawToMeter ( )

Converts the added depth-values of the Kinect ( $ Z_\mathrm{raw} \in [0, 2047) $) into meter using the formula

\[ Z_\mathrm{meter} = \frac{1}{Z_\mathrm{raw} \cdot (-0.0030711016) + 3.3309495161}, \]

which was experimentaly determined (http://nicolas.burrus.name/index.php/Research/KinectCalibration).

Consider that the z-values must be added (addZValues(int* zValues)) before before calling this function.

See also:
Step 1: Raw to meter

Field Documentation

std::vector< std::vector<float> > KinectCloud::cloud [private]

The point-cloud

std::vector< std::vector<int> > KinectCloud::cloudRgb [private]

The RGB-values

std::vector< std::vector<int> > KinectCloud::cloudRgbMapping [private]

Coordinates of a point's corresponding color at the RGB-image

std::vector<bool> KinectCloud::reconstructionSteps [private]

Reconstruction flags

std::vector<bool> KinectCloud::validValues [private]

Valid flags for each point

The documentation for this class was generated from the following files:
 All Data Structures Files Functions Variables Enumerations Enumerator
[Page Up]