Creating OpenNI '.oni' files from OpenCV/Jpeg Images

I had been thinking of tweaking OpenNI samples to work with modified data. It could be as a result of post processing on the Depth/Rgb images in Matlab or OpenCV. Mostly we record data from OpenNI in the form of '.oni' file and then want to process them in Matlab or OpenCV. For this we need to convert the images to OpenCV images and then can save to disk. Later load these images in Matlab or OpenCV and do processing and then resave them as '.oni' files to run the OpenNI samples on the modified data. 


For the first part you can either use OpenNI and convert them to OpenCV images and write them to disk. 




You can convert images to OpenCV images in the following way. 
   

         cv::Mat colorArr[3];
        cv::Mat colorImage;
        const XnRGB24Pixel* pImageRow;
        const XnRGB24Pixel* pPixel;

        imageGen.SetPixelFormat(XN_PIXEL_FORMAT_RGB24 );  //
        xn::ImageGenerator imageGen;
        imageGen.GetMetaData(imageMD);    //xn::ImageMetaData imageMD;
        pImageRow = imageMD.RGB24Data();

        colorArr[0] = cv::Mat(imageMD.YRes(),imageMD.XRes(),CV_8U);
        colorArr[1] = cv::Mat(imageMD.YRes(),imageMD.XRes(),CV_8U);
        colorArr[2] = cv::Mat(imageMD.YRes(),imageMD.XRes(),CV_8U);

        for (int y=0; y<imageMD.YRes(); y++)

       {
              pPixel = pImageRow;
              uchar* Bptr = colorArr[0].ptr<uchar>(y);
              uchar* Gptr = colorArr[1].ptr<uchar>(y);
              uchar* Rptr = colorArr[2].ptr<uchar>(y);
              for(int x=0;x<imageMD.XRes();++x , ++pPixel)

              {
                        Bptr[x] = pPixel->nBlue;
                        Gptr[x] = pPixel->nGreen;
                        Rptr[x] = pPixel->nRed;
              }
              pImageRow += imageMD.XRes();
        }
        cv::merge(colorArr,3,colorImage); 

The second way is to directly use the Matlab Toolboox for kinect and save the images from there. 

For the second part i.e how to compile these images as an '.oni' file , I used the OpenNI program  NiRecordSynthetic. So I provide a dummy .oni file as an input and then change its data according to my new depth maps and record a new .oni file out of it. This is what I do in the transformMD() function of this program

***************************************************************

   char filename[100];

        static int count = 1;

        sprintf(filename,"filename.png",count++); // here i assume that images are numbered in a sequence

        IplImage* img=0;

         img=cvLoadImage(filename,CV_LOAD_IMAGE_ANYDEPTH |      CV_LOAD_IMAGE_ANYCOLOR); // Reading 16 bit depth maps

          if(!img) printf("Could not load image file: %s\n",filename); /

      

        DepthMap& depthMap = depthMD.WritableDepthMap();

for (XnUInt32 y = 0; y < depthMap.YRes(); y++)

{

for (XnUInt32 x = 0; x < depthMap.XRes(); x++)

{

CvScalar s;

              s=cvGet2D(img,y,x); // getting the value in s 

             depthMap(x,y) = s.val[0]; // setting the value to the modified depth pixel 

}

}

cvReleaseImage(&img);

*****************************************************************************