On a recent project I had to persist image features into a database to be used on a content-based image retrieval (CBIR) system. These features included the following:

• Color - represented as histograms for each color band; each histogram consisted of 256 float numbers.
• Shape - represented as 127 Fourier coefficients (complex numbers).
• Texture - represented as an array of the top 60 peaks locations (and sign) on a 128x128 wavelet decomposition grid; the wavelet decomposition is performed in each of the three color bands.

I normally do my modeling in terms of object orientation first. I created an ImageFeatures class which had references to other classes such as Color, Shape and Texture. Each of these classes ended up holding a primitive array of float numbers for the corresponding feature data. That's all well and good. Pretty simple stuff.

To satisfy the persistence requirements I used Hibernate as the persistence framework to do the object-relational mapping to the backend database. There are many ways you could map the object structure I had into database entities. The mapping I chose resulted in one database table - IMAGE_FEAT - which held all the image feature attributes. The next step was how to map the various primitive arrays of float numbers into table columns. Here again I went with the most strightforward approach - store each primitive array as a Blob. So I ended up with a table containing eight (8) Blob columns. I could persist the image features into the database and retrieve them from the database when processing was needed. Life was good until...

... I had to fetch a set of 10,000 image features to compute the similarity index against a query image. The total query processing time would range from just under 4 minutes to almost 6 minutes. Not so good. I somehow knew that using Blobs would be bad for performance, but not this bad - Ouch!

Obviusly at this point you start thinking about in-memory caching strategies, which this is prime candidate for, because the images features once computed they don't change. So I setup a cache and performance was good as long as I had the features cached in memory. To contrast the difference, the total query processing time would now take less than 10 seconds when the 10,000 image features where cached. Still this is a system that is expected to grow into the several hundreds of thousands of images, and your in-memory cache will be limited by the amount of system memory. The cache hit % will not always be 100%.

Luckily for me, further research showed that the CBIR system could still perform at the same level with a reduced feature set, meaning, instead of using a 256-bin color histogram a 16-bin color histogram would do, the 127-coefficient Fourier shape descriptor became a 5-coefficient Fourier shape descriptor, while the texture feature remained the same. This afforded me the opportunity to rethink how to store the feature data more efficiently in the database. The resulting mapping is as follows (still using only one table):

• each Fourier coefficient is now mapped to a separate column;
• each of the three 16-bin color histogram, and each of the three 60 wavelet coefficients locations array, are now mapped to a VARCHAR field where each primitive array of float numbers is encoded (stringified) when persisted, and they are decoded back to a float array when retrieving it from the database.

The resulting table went from having 8 BLOB columns to 6 VARCHAR columns and 5 numeric columns. I know you are asking, so how did this change improved performance? Well, the answer is considerably. Now the total query processing time when fetching a set of 10,000 image features from the database ranges from 25 to 40 seconds. Not too shabby!

Blob indigestion... see ya!