# Study on Retrieval Technique of Content-Based Agricultural Scientech Multimedia Data

Xiaorong Yang, Wensheng Wang

► To cite this version:

Xiaorong Yang, Wensheng Wang. Study on Retrieval Technique of Content-Based Agricultural Scientech Multimedia Data. Third IFIP TC 12 International Conference on Computer and Computing Technologies in Agriculture III (CCTA), Oct 2009, Beijing, China. pp.77-83, 10.1007/978-3-642-12220-0_12. hal-01061717

HAL Id: hal-01061717

https://inria.hal.science/hal-01061717

Submitted on 8 Sep 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
STUDY ON RETRIEVAL TECHNIQUE OF
CONTENT-BASED AGRICULTURAL
SCIENTECH MULTIMEDIA DATA

Xiaorong Yang<sup>1,*</sup>, Wensheng Wang<sup>1,*</sup>

<sup>1</sup> Agriculture Information Institute, Chinese Academy of Agriculture sciences, No.12
Zhongguancun South St., Haidian District, Beijing 100081, P.R.China

* Key Laboratory of Digital Agricultural Early-warning Technology (2006-2010), Ministry of Agriculture, The People's Republic of China

**Abstract:** Traditional text information-based management and utilization methods can not satisfy application demands of implicit and unstructured agriculture multimedia data. For providing a readable-friendly agriculture science & technology information service, this paper proposes a relation database model oriented multimedia data method, which implements content-based multimedia data search by indexing every kind of multimedia data.

**Keywords:** Multimedia; Data Matrix; Content-Based Retrieval

# 1. INTRODUCTION

With the development of rural informatization in China, agriculture scientech information service for agriculture management departments, agriculture scientific workers, popularizing agricultural techniques workers and formers is an important content of socialist new countryside construction. Because traditional text information-based agriculture scientech information service can not satisfy users' demands, multimedia information such as pictures and videos becomes more popular due to its good intuitionism and readability. But multimedia information takes up mass storage space and hasn't semantic interpretation, text information-based management and utilization methods can be not used for the management of the implicit and

1. 国家科技支撑计划课题“农业资源利用与管理信息化技术研究与应用”（2006BAD10A06）

2. 科研院所技术开发研究专项“基于智能检索的西藏科技资源共享技术”
unstructured agriculture multimedia data. So multimedia technology should
be used to store and manage all kinds of multimedia data.

## 2. MANAGEMENT METHOD OF CONTENT-BASED AGRICULTURAL SCIENTECH MULTIMEDIA DATA

Multimedia data include three kinds: The first kind is multimedia primitive objects such as static or active images, audio files and video files. The second kind is metadata which are description information about multimedia objects. The third kind is the correlation methods between multimedia objects and their metadata. On account of multimedia data are composed of different multimedia objects, different kind of multimedia data should adopt different representation format and access methods.

### 2.1 The storage model of object-oriented relational data

Because multimedia data are high-capacity, none explanatory and nonstructural, Relational Database Management System (RDBMS) based on relational data model can not satisfy the storage demands of multimedia data. We analyzed three common data models which were object-oriented relational data model, pure object-oriented data model and relational&object-oriented data model. Being improved based on relational data model and object-oriented data model, they have limitations in managing multimedia data. On one hand, the method based on pure object-oriented data model is very difficult and lacks uniform data model and strong theory foundation. And it has poor compatibility and transplantability and doesn't support SQL statements. Besides it is poor in query optimization and view function. On the other hand, though it integrates the mature relational data model and object-oriented data model, the method based on relational& object-oriented data model can not support some object-oriented semantic. It gives up some function of object-oriented database to preserve the storage structure of RDBMS so that it reduces overall efficiency of database. And different database which are developed by different manufacturers adopt different object-oriented mechanism(W.L.Geroski et al. 1998). So we selected the object-oriented relational data model which combines object-oriented technology and database technology and applied object-oriented idea to complicated kind of data. We adopted the model and used C++ to develop application software to fulfill these functions including of data storage, management and retrieval. The storage address information of multimedia objects in a server was stored in RDBMS. For example, a real multimedia object was stored in the specified directory of a server according
to some naming rules and its keywords, text title, the address information and name were stored in RDBMS as index. We established the connection between the storage directory of multimedia objects and their keywords so that users could access multimedia objects. High-capacity multimedia objects were stored as outside data files so the optimization of database wasn't reduced. But multimedia objects could not get protection from database. So data backup should be done regularly to protect them(Hao Ying 2004).

## 2.2 Agricultural Scientech Multimedia metadata

Because agricultural scientech multimedia objects don't have interpretation, we must establish their metadata. Users can access multimedia objects which have specified features or certain contents according to the corresponding relation between multimedia objects and their metadata. Metadata are used to describe multimedia objects' content, structure, semantics, size, type, created time and so on. We created two levels metadata which were lower level vision features and higher level semantic features. Metadata about lower level vision features are those related to content. They were extracted automatically by using the feature extraction function. These metadata included the information about colors, texture, contour, shape, volume, spatial relation, movement, deformation(for example object's bending), vision object's source and features(for example source object, source event, source property, event, event property and so on) and model. For example, one person's facial features information(such as his eyes&nose's type and his hair's color) or the photography movement information(such as panning the camera and changing the focus) can be gotten. And Metadata about higher level semantic features were marked by people. Information editor described the features according to his impression such as facial expression (angry or happy) and the information related to multimedia objects such as photographer's name as background information.

We created different metadata according to different multimedia objects as follows:

### (1) Text Metadata

Text metadata were described by using text information language SGML. They were about text content description, text data presentation(text formatting, coding and compression technique), text history(modifying date), text positioning(text storage location) and so on.

### (2) Images Metadata

Images metadata were extracted by using generating algorithm which divided an image into several areas or objects to locate the objects of the image according to the image's property such as color and texture and so on.
The technique of areas growth was used to separate an image's objects. It separated an image into a group of single pixels or areas of a group of pixels whose internal parts were growing up to their boundary according with the object's boundary. Every area's property information such as grayness, color and texture was identified and assigned a value. These values formed the image's metadata by merging set operation of sets.

### (3) Video Metadata

A video is an unstructured, 2-dimensional image flow sequences. It has dynamic such as the change of the lens movement, the moving object's size and the trajectory path of video object besides common still images' features. Video data are described with act, scene, lens, frame and so on. An act is composed of a series of correlative scenes. A scene is composed of some lens. A lens is composed of a series of continuous frames. And a frame is a still image and the smallest video unit (Xu Jiuling et al. 2003).

We created three kinds of video metadata. The first kind was those related to content which were used to describe a video's primitive features such as the camera movement, light and dark of light, the object trajectory path of video frame sequences and the color chromatography curve of independent video frame. The second kind was those about content which were used to describe a video's content such as the lens' distance, the camera angle, object kind in lens and the frame's brightness, colors, texture, object kind in a frame of independent video frame. The third kind was those unrelated to content which were used to describe the whole video's features such as its created date and director's name (Yang Zhiping et al. 2002).

## 3. RETRIEVAL SYSTEM OF CONTENT-BASED AGRICULTURAL SCIENTECH MULTIMEDIA DATA

### 3.1 The Architecture of Content-based Agricultural Scientech Multimedia Data System

The retrieval system of content-based agricultural scientech multimedia data adopted this kind of method which was different from that of traditional database and approximate matching technique. Fig.1 is the system architecture map. The system includes 4 layers. The first layer is the expression Layer. Users access the portal to get agricultural scientech multimedia data. The second layer is the application layer. The application system provides retrieval service for users. After having dealt with users' request, it accesses the database and returns the retrieval result to users. The
third layer is the data layer. There are metadata database set and agricultural
scientech multimedia objects. The data association method is adopted to
create the corresponding relation between multimedia objects and their
metadata. The system finds the multimedia objects according to its metadata.
The forth layer is the supporting environment. It includes hardware such as
server, storage devices, network and software such as OS, application
middleware, database, antivirus software and so on.

<figure><img src="image_2.png" /><figcaption>Fig.1: The architecture of content-based agriculture scientech multimedia data system</figcaption></figure>

## 3.2 Retrieval Method of Content-based Agricultural Scientech Multimedia Data

Content-based retrieval method is to create multimedia object's index by
adopting its metadata. It is a kind of similarity retrieval method about
multimedia objects. The retrieval result is gotten with successive refinement
by adopting the method of approximate or partial matching. The retrieval
process is as following: firstly, the system analyzes the real content of a
multimedia object and evaluates the indicated predicate. The retrieval object
matches with query standards approximately. A corresponding characteristic
vector is given to a multimedia object based on approximate concept. That is
to say, every multimedia object is mapped to a point in attribute space. The
query range around the query point expands according to a specified amount
of deviation. Multi-dimension index structure is used to query the object
corresponding with the query point in query range. The query results
probably include the objects which don't satisfy the condition but certainly
include all objects which satisfy the condition. Then two-stage solving
technology is used to continue querying in the query results gotten by rough
query(Zhang Zhigang et al. 2007).

Different kind of multimedia object adopts different content-based
retrieval method.

(1) Content-based text retrieval method

Content-based Text retrieval method adopts full text retrieval technique
and probabilistic model namely probabilistic dependency among entrys or
between entry and document.

(2) Content-based images retrieval method

By combining pattern recognizing technology and human-machine
interactive technology, feature index is created and stored in the feature-
database according to colors, texture, shape and spatial relation o f an image.
When searching, the system analyzes an image's features to find the needed
image. The features include colors' distribution, correlation, component,
texture's structure, direction, symmetric relation, contour's shape and size,
sub-images' relationship, number and attribute, the image's approximate
description.

The retrieval system based on color features pre-treats quantitatively
image color specifications firstly. An image is indexed according to its
overall color distribution or color collection vector (CCV). When searching,
firstly the system compares the similarity of two images by using color
histogram. Then the system calculates every color's pixel. At last the image
which has the same color content can be searched.

An image is composed of different texture areas. Some images are
irregular in local areas but regular as a whole. When searching, the system
mainly measures an image's texture features such as roughness, regularity,
line similarity, contrast, direction and so on and then selects the needed
texture from samples.

The retrieval system based on shape features can search an image in
relation to its contour line by using template matching method. Users can
make an image's rough shape or contour line by hand or mapping instrument
and the system matches it with the images stored in system(Luo Deyong et
al. 2003, He Weilie 2007).

(3) Content-based video retrieval method

The content-based video retrieval system firstly treats video information
in order to restore motion information when searching. It analyzes a video's
structure and calculates frame difference by detecting lens border and
different lens' color and brightness, and then divides a video into some basic
lens units. The system selects the key frames to represent complicated video
by using frame average method and Histogram average method. By
extracting every lens' features it creates index to form feature space, then
depends on feature space to compare lens' content. At last according to these
features it combines the lenses which have similar contents. When searching,
the system matches feature according to user's certain requirements and
searches again according to user's feedback until user gets satisfying results.
This is a refinement process step by step(Wang Weihua et al. 2007).

4. CONCLUSION

Multimedia object takes up mass storage space and hasn't semantic
interpretation, traditional text information-based management and utilization
methods can be not used for the management of the implicit and
unstructured agriculture multimedia data. The object-oriented relational data
model was adopted to manage multimedia data efficiently. High-capacity
multimedia objects were stored as outside data files so the optimization of
database didn't be reduced. Agricultural scientech multimedia metadata were
created to relate multimedia object and its index so that the system can
access the needed data. Retrieval system of agricultural scientech
multimedia data was developed to fulfill the storage and access of content-
based multimedia data efficiently.

With the rapid development of digital communication network and
increasing capacity of computer processing, it is practicable and economic to
produce and utilize multimedia data. Transmitting the agricultural scientech
multimedia data gives new content for the agricultural scientech information
service and promotes the development of rural informatization in China.

ACKNOWLEDGEMENTS

The work is supported by the Academy of Science and Technology for
Development fund project “intelligent search-based Tibet science &
technology information resource sharing technology”, National Science and
Technology Support Program (Grant No. 2006BAD10A06), special project of the Ministry of Science and Technology "TD-SCDMA based application development and demonstration validation in agriculture informationization", and the special fund project for Basic Science Research Business Fee, AII (No.2009J-06).

## REFERENCES

[America]W.L.Geroski et al. Management technical manual of multimedia information, Cambridge University Press, 1998

Hao Ying. Treatment of multimedia data in ORDBMS, Journal of Beijing Institute of Civil Engineering and Architecture, 2001,17(2):77~79

Xu Jiuling, Xiao Huizhen. Study on metadata, Journal of Information Theory and Practice, 2003,26(2):163~166

Yang Zhiping, Feng Sumei. Storage and retrieval of metadata in multimedia database, Journal of Chongqing Teachers College, 2002,19(1):14~19

Zhang Zhigang. Study on multimedia database and its content-based retrieval method, Journal of Information Technology and the Informationization, 2007,2: 61~62

Luo Deyong, Ming Hai. Study on content-based multimedia data retrieval method in digital library, Journal of Information Exploring, 2003(1): 21~23

He Weilie. Study on content-based multimedia data retrieval technology, Journal of TV University Engineering, 2007,2:59~61

Wang Weihua, Wang Weiqing. Content-based multimedia data retrieval technology, Journal of Computer Project and Design, 2007,10: 2373_2375