Large data set storage and display for electronic spreadsheets applied to machine vision

An electronic spreadsheet is improved for machine vision applications by incorporating single method objects, and a data display buffer position beneath a variably transparent spreadsheet grid. The data display buffer works in cooperation with the single method objects to selectively display stored data sets in superimposed relationship with the variably transparent spreadsheet grid. The resulting user-interface is particularly advantageous for machine vision applications, and other applications using large data sets.

FIELD OF THE INVENTION

The invention relates to user interfaces for computer programs, and more particularly to user interfaces in the field of machine vision.

BACKGROUND OF THE INVENTION

In the field of factory automation, the high cost of implementing machine vision technology represents a significant barrier to the wide-spread adoption of industrial image sensors. Traditionally, a large part of this cost has been attributed to the expense of software development. Programmers with specialized machine vision experience are often required to accomplish ostensibly simple machine vision tasks. Although many attempts have been made to simplify application development, there has been no clear solution to the machine vision ease-of-use problem.

The superior ease-of-use characteristics of electronic spreadsheets are well known, especially in such fields as office automation. The use of electronic spreadsheets in the field of machine vision is also known, although in the past, the spreadsheet metaphor could not be adequately exploited for such applications. One reason for this is that machine vision programming tasks typically involve the manipulation of images and other large data sets, a task poorly suited to conventional electronic spreadsheets.

A standard spreadsheet consists of a rectangular grid of cells, each cell representing a single value that is, in turn, a function of some number of other cells in the grid. The standard electronic spreadsheet is suited for implementing formulas that require a relatively small number of input values and generate a single scalar output. In contrast, certain machine vision functions are parameterized by arguments representing a million or more pixel values. Since there is no facility in a standard electronic spreadsheet that enables a single cell to represent multiple values, the standard spreadsheet model is clearly not suited to support functions that operate on images and other large data sets.

In the fields of scientific visualization and computer graphics, the term “spreadsheet” has been loosely applied to certain programs that manage multiple images and other data representations in a two-dimensional rectangular grid within a bounding window. These programs, typically designed for graphical editing and array processing, essentially, allow individual data cells to be formatted as data arrays. However, in order to accommodate data sets of varying size, these interfaces generally abandon the strict two-dimensional grid relationship between cells that is the essence of the spreadsheet paradigm. In fact, most of these programs are unrecognizable as spreadsheets, and are generally unsuited for machine vision.

SUMMARY OF THE INVENTION

The invention provides methods for adapting electronic spreadsheets to applications in the field of industrial machine vision. The invention also substantially preserves the fundamental characteristics of conventional electronic spreadsheets. The invention also makes efficient use of a given display area and resolution, using for example, a semitransparent spreadsheet superimposed over an image of the contents of a data buffer. Further, the invention facilitates and simplifies the manipulation of images and other large data sets, while preserving the ease-of-use, and other advantages and benefits, of the spreadsheet paradigm.

The invention exploits a new class of object-oriented spreadsheet functions that incorporate internal data buffering. In addition, the invention provides methods by which the graphical representation of buffered data sets may be displayed simultaneously with a spreadsheet grid, preferably as a graphics layer positioned beneath a variably transparent spreadsheet grid. A split-screen and dual screen approach can also be employed for displaying the buffered data sets alongside the electronic spreadsheet.

In a preferred embodiment, a conventional spreadsheet is extended to include data buffering functions, herein referred to as “single method objects”, because from the perspective of the spreadsheet programmer, there is only one public member function, i.e., only one public method. For example, in accordance with the invention, the familiar “search” tool of machine vision is the public member function of the search object, which includes a search function, as well as data associated with the search function.

A single method object can be instantiated and assigned to a spreadsheet cell within a variably transparent grid superimposed on a graphical display buffer, the contents of the graphical display buffer being determined in accordance with the currently selected cell. If an empty cell, or a cell containing a conventional spreadsheet formula is selected, the image last acquired is stored and displayed in the underlying graphical display buffer. If a cell containing a single method object is selected, a graphical representation of the object data members is stored in the underlying buffer and displayed.

The foregoing methods selectively extend data dimensionality to meet the practical requirements of machine vision processing, while still retaining the fundamental characteristics and ease-of-use of a conventional electronic spreadsheet.

DETAILED DESCRIPTION

Referring toFIG. 1, a preferred spreadsheet user interface for machine vision applications includes a monitor11connected by cable to a computer12. The computer12includes image acquisition, processing, and graphical display capability. A camera and lens assembly13has an object surface15within its field of view, and is connected to the computer12. Also, connected to the computer12is a user input device, such as a game controller14, or a standard keyboard (not shown).

The screen of the monitor11depicts a semitransparent spreadsheet16superimposed on an image and graphics layer17to form a composite display of the invention. The transparency of the semitransparent spreadsheet16can be adjusted by the user. The semitransparent spreadsheet16can also be displayed in a split-screen arrangement wherein the image and graphics layer17is displayed on one portion of the screen of the monitor11, and the semitransparent spreadsheet16, which may be adjusted so as to no longer be transparent, can be displayed in another portion of the screen of the monitor11. Alternatively, the image and graphics layer17can be displayed on the screen of the monitor11, and the semitransparent spreadsheet16, which may be adjusted so as to no longer be transparent, can be displayed on the screen of a second monitor (not shown).

FIG. 2shows the superposition combination of the contents of data display buffer21and the semitransparent spreadsheet overlay22to form a composite display23. Not shown are the split-screen combination, and the dual-screen combination of the contents of data display buffer21and the semitransparent spreadsheet overlay22.

FIGS. 3,4,5, and6illustrate the effect of spreadsheet cell selection on the underlying data display buffer. The contents of the underlying data display buffer changes in accordance with the particular cell that is currently selected. Also, the contents of the formula line changes in accordance with the particular cell that is currently selected. Note that only the data contents of the selected cell is displayed; all of the other data associated with the objects underlying the other cells remains invisible to the user. By contrast, all of the data in a standard spreadsheet is visible to a user, including a large standard spreadsheet, notwithstanding the need to scroll over the large spreadsheet.

In monitor31ofFIG. 3, spreadsheet focus (highlight)33, indicated by the frame border (or any selection highlighting), is on cell A1. Formula line32indicates that cell A1is assigned the single method object ACQUIRE, which contains an image buffer resulting from its image acquisition method. The underlying data display buffer34shows the content of the image buffer associated with ACQUIRE.

In monitor41ofFIG. 4, spreadsheet focus43is on cell A2. Formula line42indicates that cell A2is assigned the single method object PROCESS which contains the image buffer that results from the processing a rectangular region45in the image associated with cell A1. The other four arguments of the object PROCESS represent the coordinates of the upper-left corner, and the height and width, of the rectangular region45in the image associated with cell A1. The underlying data display buffer44shows the content of the image buffer associated with the processed image.

In monitor51ofFIG. 5, spreadsheet focus53is on cell A3. Formula line52indicates that cell A3in this example is assigned the single method object HISTEXTRACT which contains a histogram array extracted from a rectangular region55in the processed image assigned to cell A2. The other four arguments of the object PROCESS represent the coordinates of the upper-left corner, and the height and width, of the rectangular region55in the image associated with cell A2. The underlying data display54shows the content of the input image buffer plus a graphical representation of the histogram array associated with the content of the input image buffer.

In monitor61ofFIG. 6, spreadsheet focus63is on cell A4. Formula line62indicates that cell A4in this example is assigned a contrast value computed from the histogram array assigned to cell A3. The underlying data display buffer64shows the content of the acquired image.

The forgoing illustrates how single method objects co-operate with a variably transparent spreadsheet overlaid on the contents of a data display buffer so as to efficiently and elegantly extend the conventional spreadsheet paradigm to accommodate images and other large data sets.

Other modifications and implementations will occur to those skilled in the art without departing from the spirit and the scope of the invention as claimed. Accordingly, the above description is not intended to limit the invention except as indicated in the following claims.