Started working on segmentation with Felzenszwalb's method with K-nearest graph in (hue,x,y) feature space. It seems that just hue is not enough, perhaps identifying using Lab color space might be more interesting. Also it seems to produce very fragmented segmented. A better idea might be to first use Felzenszwalb's method with a simple 4-connected or 8-connected grid graph, then fuse components which are close enough in hue in a second step.
Got a half-decent segmentation algorithm. Still requires some tuning and some parameters to be automatically set so it gives good results for all characters.
After a bit of tweaking with hue, got a reasonably good looking segmentation method. Will now try to classify these segmentations using segment matching method.
Now got the algorithm running on the entire data set, with a 0.3 recognition rate - which is much better than random but still not good by any stretch. The segmentation algorithm seems to be pretty good, as it still gives 100% for lupin and luffy. But trickier characters like Rei seem to introduce outliers all over the place. Classification will need to be robust to those. Furthermore I'll identify exactly what provokes characters to be misclassified, I suspect that segments which are cut in multiple parts in some image may be a big source of error. As the number of segment is small, it might be practical to consider not just 1 to 1 relations between segmentations, but 1 to k for some small k as well - the combinatorial explosion should still be manageable.
Changed the similarity function to a symmetric one. Now yields a 35% recognition rate with leave one out cross validation. The method seems to mostly confuse characters with similar color schemes. I see 2 ways to improve classification:
- a better classification method which is more robust to outliers, using K-nn with larger K or some semi-supervised spectral method.
- a similarity function which takes into account how well the matching preserves image structure.