Good Day!
I employ DGM for medical image segmentation. Thanks in advance for providing such useful toolbox. Do you have any dense crf version for DGM?
I want to improve it so that it becomes a fully connected graphical model as Philip Krahenbuhl's paper. I wonder if you kindly please guide me in this point.
Dense CRF with DGM
-
- Posts: 12
- Joined: Sun Dec 10, 2017, 03:31
- Contact:
Re: Dense CRF with DGM
Thank you for the interest to the DGM library.
I have revised the dense-CRF Code, to make it work with OpenCV and DGM library. You can download it here: https://github.com/ereator/Dense-CRF
Currently I work on integrating the DGM library into the Dense-CRF. For that you Need to extract the potential Matrices. For example, now it is possible to save and read These Matrices in DGM as: http://research.project-10.de/dgmdoc/a0 ... 2cf6f688a2 and http://research.project-10.de/dgmdoc/a00618.html
I hope this Information will help you.
I have revised the dense-CRF Code, to make it work with OpenCV and DGM library. You can download it here: https://github.com/ereator/Dense-CRF
Currently I work on integrating the DGM library into the Dense-CRF. For that you Need to extract the potential Matrices. For example, now it is possible to save and read These Matrices in DGM as: http://research.project-10.de/dgmdoc/a0 ... 2cf6f688a2 and http://research.project-10.de/dgmdoc/a00618.html
I hope this Information will help you.
-
- Posts: 12
- Joined: Sun Dec 10, 2017, 03:31
- Contact:
Re: Dense CRF with DGM
So, if I include serialize.h in Demo train.cpp of DGM and use "to", I can save Potential matrices in that dat format. Further I can use it in the modified version of densecrf.
How long it takes to produce DGM with Denscrf? I wonder if you kindly please share with me the source code that I have to write in DGM to save the matrices.
How long it takes to produce DGM with Denscrf? I wonder if you kindly please share with me the source code that I have to write in DGM to save the matrices.
Re: Dense CRF with DGM
So, if I include serialize.h in Demo train.cpp of DGM and use "to", I can save Potential matrices in that dat format. Further I can use it in the modified version of densecrf.
That is right.
How long it takes to produce DGM with Denscrf?
Do you mean how long will it take to implement DenseCRF into the DGM library? I plan to build it in version 1.6.0. With my current speed of the development it will be at the end of this year.
Or do you mean how long the calculation process takes? The calculation and saving the potentials in DGM takes about 1 – 10 seconds, depending on the image size. The DenseCRF inference takes about 1 – 5 seconds.
I wonder if you kindly please share with me the source code that I have to write in DGM to save the matrices.
It is only one string of code. For example in Demo Train:
Code: Select all
#include "DGM/serialize.h"
...
// ==================== STAGE 3: Filling the Graph =====================
Timer::start("Filling the Graph... ");
Mat nodePotentials = nodeTrainer->getNodePotentials(test_fv);
Serialize::to("fileName.dat", nodePotentials); // <<<< THIS LINE
graph->setNodes(nodePotentials);
graph->fillEdges(edgeTrainer, test_fv, params, params_len);
Timer::stop();
-
- Posts: 12
- Joined: Sun Dec 10, 2017, 03:31
- Contact:
Re: Dense CRF with DGM
Thank you for the response.
I downloaded two version of DGM:
This is my code:
When I write and it works.
Would you please inform me how to fix this error?
I downloaded two version of DGM:
- The old one DGM v.1.5.1 which does not include serialize.h. In this program, I use a feature vector which is an array of 36 matrices for image classification. It works properly.
- However, in the DGM-master (with opencv-3.2.0), which include serialize.h, when I use the above mentioned feature vector, the program does not work and has the following error:DGM_ASSERT_MSG(!isnan(node->Pot.at<float>(s, 0)), "The lower precision boundary for the potential of the node %zu is reached.\n
in the MessagePassing.cpp
I explore all the features to be sure whether they are between 0,255. My code works in DGM v.1.5.1 when I use all 36 features. However, when I just use each element of the array of 36 matrices, i.e. one matrix as a feature vector or some of them for classification, DGM-master works.
This is my code:
Code: Select all
// ----------- training image feature vector -----------
Mat FeatureSets;
vec_mat_t channels;
cv::Ptr<cv::ml::TrainData> raw_data = cv::ml::TrainData::loadFromCSV("E:\\zghassabi\\Project1-DIE\\Dataset\\image 1\\All_features.csv", 0);
cv::Mat data1 =raw_data->getSamples();
cv::Mat data2 = raw_data->getResponses();
cv::hconcat(data1, data2, data1);
cv::Mat data;
data1.convertTo(data, CV_8UC1);
int i = 0;
int colsize = 1024;
int rowsize = 61;
for (i =0; i < data.cols; i += colsize) {
cv::Mat m = data.colRange(i, i + colsize);
channels.push_back(m);
imshow("Image", m);
cvWaitKey(0);
}
merge(channels, FeatureSets);
Mat train_fv = FeatureSets; resize(train_fv, train_fv, imgSize, 0, 0, INTER_LANCZOS4);
When I write
Code: Select all
for (i =0; i < 5120; i += colsize)
Code: Select all
nfeatures=5
Would you please inform me how to fix this error?
Re: Dense CRF with DGM
The error in MessagePassing.cpp is related to the inference stage. Having the current information, you shared with me, it is difficult to say, where exactly the problem lies. It might be in the node training phase, or data preparation phase. Could you please provide some additional information, in particular to the following questions:
Ideally, I could help you more effectively, if I could reproduce the error. As I said it is difficult to define the source of this bug without debugging the code. (I assume, that since it works in DGM v.1.5.1 and does not work in DGM v.1.5.2, it is a bug).
- Do you save and/or load the training data in-between the training and classification stages?
- The error occurs on the first iteration or after some iteration?
- Which inference method do you use? Have you tried running different inference methods?
- Could you please check that the node potentials, used in the graph filling stage are not empty?
- Which node and which edge training models do you use?
- You wrote that you have checked the features. Have you also checked the responses in In particular for the class, starting from which, the bug appears?
Code: Select all
Mat data2 = raw_data->getResponses();
Ideally, I could help you more effectively, if I could reproduce the error. As I said it is difficult to define the source of this bug without debugging the code. (I assume, that since it works in DGM v.1.5.1 and does not work in DGM v.1.5.2, it is a bug).
-
- Posts: 12
- Joined: Sun Dec 10, 2017, 03:31
- Contact:
Re: Dense CRF with DGM
Thank you very much for your response.
regarding first question:
regarding second question:
When I choose the below three sections in the node-training-model, the decoding part does not work properly. It seems that after several iteration the error appears.
section 1: Gaussian Mixture model
section 3: nearest neighbor
section 5: Microsoft random Forrest.
regarding third question:
I explored different combination of node taring models with edge training models. It seems that when the number of features is more than 5, GMM, k-nn, mRF do not work. I tested all of these combination with 1D feature . All of them work when the feature is 1D or 3D.
nodePotentials is not empty.
node-training-model: OpenCV Gaussian Mixture Model
edge-training-model: Concatenated Model
the above models do not produce errors for 5-D feature.
When I choose node training model as
section 1: Gaussian Mixture model
section 3: nearest neighbor
section 5: Microsoft random Forrest.
and when I use a feature matrix with higher dimensions.
regarding first question:
Do you save and/or load the training data in-between the training and classification stages?
Code: Select all
nodeTrainer->train();
nodeTrainer->save("E:\\zghassabi\\Project1-DIE\\Dataset\\image 1\\nodetrainn.dat");
nodeTrainer->reset();
nodeTrainer->load("E:\\zghassabi\\Project1-DIE\\Dataset\\image 1\\nodetrainn.dat");
edgeTrainer->train();
Timer::stop();
// ==================== STAGE 3: Filling the Graph =====================
Timer::start("Filling the Graph... ");
// CV_32FC(nStates) <- CV_8UC(nFeatures);
Mat nodePotentials = nodeTrainer->getNodePotentials(test_fv); // Classification: CV_32FC(nStates) <- CV_8UC(nFeatures)
Serialize::to("E:\\zghassabi\\Project1-DIE\\Dataset\\image 1\\fileName.dat", nodePotentials);
graph->setNodes(nodePotentials); // Filling-in the graph nodes
graph->fillEdges(edgeTrainer, test_fv, params, params_len); // Filling-in the graph edges with pairwise potentials
Timer::stop();
regarding second question:
The error occurs on the first iteration or after some iteration?
When I choose the below three sections in the node-training-model, the decoding part does not work properly. It seems that after several iteration the error appears.
section 1: Gaussian Mixture model
section 3: nearest neighbor
section 5: Microsoft random Forrest.
regarding third question:
Which inference method do you use? Have you tried running different inference methods?
I explored different combination of node taring models with edge training models. It seems that when the number of features is more than 5, GMM, k-nn, mRF do not work. I tested all of these combination with 1D feature . All of them work when the feature is 1D or 3D.
Could you please check that the node potentials, used in the graph filling stage are not empty?
nodePotentials is not empty.
Which node and which edge training models do you use?
node-training-model: OpenCV Gaussian Mixture Model
edge-training-model: Concatenated Model
the above models do not produce errors for 5-D feature.
You wrote that you have checked the features. Have you also checked the responses inIn particular for the class, starting from which, the bug appears?Code: Select all
Mat data2 = raw_data->getResponses();
When I choose node training model as
section 1: Gaussian Mixture model
section 3: nearest neighbor
section 5: Microsoft random Forrest.
and when I use a feature matrix with higher dimensions.
-
- Posts: 12
- Joined: Sun Dec 10, 2017, 03:31
- Contact:
Re: Dense CRF with DGM
I attach also the files:
the size of features is 61 x 1024 x 36
I attached the modified source code of demo train, too.
Using these files, you can see the error.
sorry for inconvenience.
- input image, its ground truth, a text file of features for input image (for training)
- test-image, a text file of features for test image ( for testing)
the size of features is 61 x 1024 x 36
I attached the modified source code of demo train, too.
Using these files, you can see the error.
sorry for inconvenience.
- Attachments
-
- All_features_testimage.csv
- (7.43 MiB) Downloaded 1435 times
-
- All_features.csv
- (6.59 MiB) Downloaded 1498 times
-
- dgm-demo train.txt
- (13.33 KiB) Downloaded 1495 times
-
- mask_DABAB8D0.tif (37.14 KiB) Viewed 20385 times
-
- DABAB8D0-gt4.tif (1.63 KiB) Viewed 20385 times
-
- DABAB8D0-Adjacentl.tif (61.74 KiB) Viewed 20385 times
Re: Dense CRF with DGM
I have tried out your code with 5 and 36 features and could not reproduce the bug. In both cases the code produces results. I have used the latest version of the DGM library from GitHub with OpenCV v.3.3.0.
However, I think that there might be 2 reasons for the bug:
I attach also the code, that generates these results: I hope that will help you.
P.S.
I have noticed that the features data from the .csv files has 61 x 37887 samples and 61 x 1 responses. So after the concatenation you have 61 x (37*1024) data matrix, where the last 1024 values in every row are a mixture between samples and responses. Since, you use only 36 first features, it should not be a problem, however, that data container looked a bit strange to me.
However, I think that there might be 2 reasons for the bug:
- OpenCV v.3.2.0 had a bug with the data serialization:
https://github.com/Project-10/DGM/issues/8,
https://github.com/opencv/opencv/issues/9125.
But this bug was confirmed only for Random Forest classifier. Maybe it is also relevant for the ml::TrainData::loadFromCSV() function.
So I have disabled the nodeTrainer->save() / nodeTrainer->load() functionality. - When using a lot of features, CTrainNodeCvGMM class may produce very small potentials. And the small potentials in its turn may lead to the “The lower precision boundary for the potential of the node %zu is reached.” problem. So, I would suggest to use another node training model, for example k-nearest neighbors: CTrainNodeCvKNN
I attach also the code, that generates these results: I hope that will help you.
P.S.
I have noticed that the features data from the .csv files has 61 x 37887 samples and 61 x 1 responses. So after the concatenation you have 61 x (37*1024) data matrix, where the last 1024 values in every row are a mixture between samples and responses. Since, you use only 36 first features, it should not be a problem, however, that data container looked a bit strange to me.
-
- Posts: 12
- Joined: Sun Dec 10, 2017, 03:31
- Contact:
Re: Dense CRF with DGM
Thank you very very much for your kind and help!
I used OpenCV v.3.3.0 and it works properly on my computer, too. Thank you for improving the code.
I used CRF_output_CvKNN_f36.tif: Using CTrainNodeCvKNN and CTrainEdgePotts and 36 features. I also save the potential matrices in filename.dat to use in the densecrf. Would you please help me in modyfing your code for my images? I have nStates = 4. I can not understand the upper bound of m = 21 and i = 20. This is because you have 21 nStates? If I change nStates to 4, can I make comment this part:
why do you wrote the above part?
I attached you my code. The result is worse than DGM.
Thank you for your help.
I used OpenCV v.3.3.0 and it works properly on my computer, too. Thank you for improving the code.
I used CRF_output_CvKNN_f36.tif: Using CTrainNodeCvKNN and CTrainEdgePotts and 36 features. I also save the potential matrices in filename.dat to use in the densecrf. Would you please help me in modyfing your code for my images? I have nStates = 4. I can not understand the upper bound of m = 21 and i = 20. This is because you have 21 nStates? If I change nStates to 4, can I make comment this part:
Code: Select all
if (m == 17) continue;
for (word i = 11; i <= 20; i++) {
if (m == 6 && i == 17) continue;
if (m == 18 && i == 9) continue;
why do you wrote the above part?
I attached you my code. The result is worse than DGM.
Thank you for your help.
- Attachments
-
- solution-DensCRF_output.tif (5.43 KiB) Viewed 20344 times
-
- dense_inference.cpp
- (8.06 KiB) Downloaded 1379 times
Return to “Direct Graphical Models”
Who is online
Users browsing this forum: No registered users and 8 guests