System and method to configure distributed measuring devices and treat measurement data

A method to configure a measuring device is provided. In other aspects, a server and a non-transitory computer readable storage medium are provided. The method includes receiving a configuration request that specifies configuration criteria for configuring a measuring device to perform a test, wherein the configuration criteria includes configuration instructions, analysis instructions, and visualization instructions, and configuring the measuring device to perform measurements and output measurement data in accordance with the configuration instructions. The method further includes receiving the measurement data output by the measuring device, storing the measurement data received, processing the measurement data stored based on the analysis instructions, and outputting visualization data corresponding to the processed measurement data to a display device based on the visualization instructions.

FIELD OF THE INVENTION

The present disclosure relates to configuration of distributed measuring devices and treatment of measurement data, and in particular to a social platform for sharing configuration criteria and output measurement data.

BACKGROUND OF THE INVENTION

Currently in the art, engineers, such as network support engineers, have access to limited options for remotely configuring measuring devices that measure network characteristics. When the limited configuration options are not adequate, the measuring devices can be configured manually. Additionally, aggregation of data generated by measuring devices is limited due to access and compatibility issues. While this approach may have been considered adequate until the present time, there is a need for flexibility in configuration of measuring devices and treatment of measurement data output by the measuring devices, and expanded collaboration in sharing configuration information and treatment of measurement data.

SUMMARY OF THE INVENTION

The purpose and advantages of the illustrated embodiments will be set forth in and apparent from the description that follows. Additional advantages of the illustrated embodiments will be realized and attained by the devices, systems and methods particularly pointed out in the written description and claims hereof, as well as from the appended drawings.

In accordance with a purpose of the illustrated embodiments, in one aspect, a server having at least one processing device is provided. The processing device is configured to receive a configuration request that specifies configuration criteria for configuring a measuring device to perform a test. The configuration criteria include configuration instructions, analysis instructions, and visualization instructions. The processing device is further configured to configure the measuring device to perform measurements and output measurement data in accordance with the configuration instructions, receive the measurement data output by the measuring device, store the measurement data received, process the measurement data stored based on the analysis instructions, and output visualization data corresponding to the processed measurement data to a display device based on the visualization instructions.

In another aspect, a method is provided to configure a measuring device. The method includes receiving a configuration request that specifies configuration criteria for configuring a measuring device to perform a test, wherein the configuration criteria includes configuration instructions, analysis instructions, and visualization instructions, and configuring the measuring device to perform measurements and output measurement data in accordance with the configuration instructions. The method further includes receiving the measurement data output by the measuring device, storing the measurement data received, processing the measurement data stored based on the analysis instructions, and outputting visualization data corresponding to the processed measurement data to a display device based on the visualization instructions.

In a further aspect, a non-transitory computer readable storage medium and one or more computer programs embedded therein are provided. The computer programs include instructions, which when executed by a computer system, cause the computer system to receive a configuration request that specifies configuration criteria for configuring a measuring device to perform a test, wherein the configuration criteria include configuration instructions, analysis instructions, and visualization instructions. The instructions further, when executed, cause the computer system to configure the measuring device to perform measurements and output measurement data in accordance with the configuration instructions, receive the measurement data output by the measuring device, store the measurement data received, process the measurement data stored based on the analysis instructions, and output visualization data corresponding to the processed measurement data to a display device based on the visualization instructions.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

The present disclosure is now described more fully with reference to the accompanying drawings, in which an illustrated embodiment of the present disclosure is shown. The present disclosure is not limited in any way to the illustrated embodiment as the illustrated embodiment described below is merely exemplary of the disclosure, which can be embodied in various forms, as appreciated by one skilled in the art. Therefore, it is to be understood that any structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative for teaching one skilled in the art to variously employ the present disclosure. Furthermore, the terms and phrases used herein are not intended to be limiting but rather to provide an understandable description of the disclosure.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present disclosure, exemplary methods and materials are now described. It must be noted that as used herein and in the appended claims, the singular forms “a”, “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a stimulus” includes a plurality of such stimuli and reference to “the signal” includes reference to one or more signals and equivalents thereof known to those skilled in the art, and so forth.

It is to be appreciated the embodiments of this disclosure as discussed below are preferably a software algorithm, program or code residing on computer useable medium having control logic for enabling execution on a machine having a computer processor. The machine typically includes memory storage configured to provide output from execution of the computer algorithm or program.

As used herein, the term “software” is meant to be synonymous with any code or program that can be in a processor of a host computer, regardless of whether the implementation is in hardware, firmware or as a software computer product available on a disc, a memory storage device, or for download from a remote machine. The embodiments described herein include such software to implement the equations, relationships and algorithms described below.

Turning now descriptively to the drawings, in which similar reference characters denote similar elements throughout the several views,FIG. 1depicts a block diagram of an exemplary measurement configuration system100in which illustrated embodiments of the present disclosure may be implemented.

FIG. 1shows the measurement configuration system100includes an exemplary measurement configuration server102in communication with a plurality of measuring devices104, a storage system106, and a plurality of client devices108. Communication between the measurement configuration server102and the measuring devices104, storage system106, and client devices108can be direct or via one or more networks110, such as a LAN, WAN, intranet or the Internet using communication links112. The communication links112can be wired, wireless, or a combination thereof.

The measurement configuration server102includes one or more computing devices that execute programmable software modules for performing operations including receiving a configuration request from a client device108to configure at least one measuring device104, configuring the measuring device to perform measurements and output measurement data in accordance with the configuration request, receiving measurement data output by the measuring device in accordance with the configuration request, processing the measurement data in accordance with the configuration request, and configuring the processed measurement data for display in accordance with the configuration request.

The measuring devices104and client devices108include storage devices that store programmable instructions and computing devices that execute the programmable instructions in accordance with performance of methods of the disclosure as described. The measuring devices104can include sensors and/or computing devices that measure properties or characteristics and generate measurement data.

Examples of measuring devices104include network measurement tools, such as that measure network performance metrics, such as throughput, round-trip-time (RTT), frames per second (f/s); measurement tools that measure environmental and physical properties such as temperature, humidity, pressure, velocity, acceleration, properties of light and sound; measurement tools that measure system properties such as memory usage, central processing unit (CPU) utilization, network interface information, internal temperature, etc. The measuring devices104can be stationary, portable, and/or mobile. In embodiments, one or more of the measuring devices104can be virtual measuring devices. In embodiments, the measuring devices104can be web crawlers or bots that move through one or more networks being monitored and detect and measure or use characteristics of information transmitted via the network(s) being monitored, such as addresses, content, data rates, etc. Characteristics of content that can be measured can include, for example, occurrence of selected words, images, colors, etc.

Examples of client devices108include stationary or mobile computing devices such as personal computers, tablets, and smartphones. The client devices108include user input devices for a user to enter a configuration request and display devices to display visualization results, such as in a graphical user interface (GUI). User input can be acquired, for example, via user input via a GUI using a mouse, keyboard, touch screen, etc.

Referring toFIG. 2, an exemplary flow diagram is shown between software modules executed by the one or more processing devices of the measurement configuration server102and the measuring devices104, the storage system106, and the client devices108. The measurement configuration server102accesses and executes software modules, which can include, for example, a user configuration interface202, a user visualization interface (UI)204, a configuration module206, an analysis module210, an aggregation module212, and a deployment module214.

The storage system106includes storage devices to store data and one or more processing devices for executing a management system to store, access, update, retrieve, and otherwise manage the data being stored. The storage system106can include one or more databases, such as a configuration database220, a measurement database222, and an aggregation database224that use the same or separate management systems for managing data.

The user configuration interface202receives configuration requests transmitted from client devices108. The configuration request identifies one or more measuring devices104and can include configuration criteria having at least one of configuration instructions, deployment instructions, pre-analysis instructions, analysis instructions, visualization instructions, and/or aggregation instructions. The user configuration interface202can provide feedback to the user when the measuring devices104are configured. The configured measuring devices104can be activated to start or end performance of a test by a user action or automatically, such as in response to a condition (e.g., scheduled times, amount of measurement data generated, content of measurement data generated, etc.)

The user visualization interface204receives visualization data and/or alert data from the analysis module210and/or aggregation module212and provides the visualization and/or alert data to the client devices108. The alert data instructs the client device108to issue an alert notification, such as an audio, displayable, or vibratory alert notification.

The user configuration interface202and the user visualization interface204can include an application program interface (API) that provides an interface between the measurement configuration server102and the client devices108. Using the API, the user configuration interface202can interact with the configuration database220to store, access, update, and retrieve configuration requests, and with the configuration module206to provide configuration request identification to the configuration module206so that the configuration module206can access a corresponding configuration request from the configuration database220. Using the API, the user visualization interface204can provide visualization data to the client device108in response to a configuration request by interacting with the storage system106to access the corresponding configuration request, the measurement database222and aggregation database224, the analysis module210, and aggregation module212, to access data to be included in the visualization data.

The configuration module206can be for example a software module, such as an API, that accesses configuration data from the configuration database220and uses the accessed configuration data to configure, provision, and instruct the measuring devices104. The configuration module206can receive from the user configuration interface notification and identification of a received configuration request and accesses the associated configuration instructions from the configuration database220. Alternatively, the configuration module206can receive the configuration instructions from the user configuration interface202.

The configuration module206communicates with the measuring devices104, including uploading to a measuring device104specified in a communication request a configuration script specified by configuration instructions included in the configuration request. The configuration script can be included in or identified by the configuration request. The configuration module206can access the identified configuration script from the configuration database220.

In addition, the configuration module206receives measurement data output by the measuring device104and performs pre-analysis processing to the measurement data in accordance with pre-analysis instructions included in the communication request.

The configuration script specified by the configuration instructions can include programmable instructions which, when executed by the processing device of the measuring device104, configure the measuring device104to perform operations. Operations to be performed by the measuring device104can include detecting and/or measuring at least one property and outputting corresponding measurement data. A property being measured can include, for example, a physical property (e.g., temperature, current, pressure, etc.) or a characteristic of data (e.g., data content, network traffic metrics, etc.) or a characteristic of the computing environment (e.g., memory usage, CPU utilization, wireless interference, etc.). The measurement data can be output in a predetermined format that can be parsed and processed by the measurement configuration server102. Examples of formats for measurement data output by the measuring device104include a format that uses a JSON data structure having key-value pairs. Each key-value pair can include a description of the property that was measured paired with the corresponding measurement value (e.g. {“temperature”: 72, “humidity”: 0.57}). In the case of failure, the data structure can also be used for troubleshooting and debugging by including verbose details of a malfunction incident (e.g., {“temperature”: 73, “humidity”: null, “sensorFailure”: “Humidity Failure: Sensor GPIO pin 4 is disconnected” }). Additional constraints to the predetermined format can include, for example, using only simple key/value objects, use of nested objects is forbidden, outputting numeric values only, using predetermined naming conventions for metrics being measured, etc.

The pre-analysis instructions can specify fields of the measurement data to process and/or store, wherein unspecified fields of the measurement data can be ignored. For example, the measuring device104may be a virtual probe installed on a network that can detect source and/or destination addresses, however the analysis instructions specify processing only source address information, but ignoring destination address information. In this case, only the source address information will be analyzed and stored. Additionally, the pre-analysis instructions can specify statistical aggregation of measurement data to perform before the measurement data is stored in the measurement database222. For example, the statistical aggregation can specify determining during pre-analysis a minimum, maximum, mean, 3 dB level, or 6 dB level per a specified interval, wherein the interval can be specified as a time or frequency interval. The results of the pre-analysis generated by the configuration module206can be stored in the measurement database222without storing the raw measurement data output by the by the measuring device104. Alternatively, both the pre-analysis results and the raw measurement data can be stored in the measurement database104.

The configuration module206can store in the measurement database222the pre-analysis results and/or the raw measurement data received from the measuring devices104.

The analysis module210analyzes measurement data output by the measuring devices104based on analysis instructions, and generates visualization data for displaying the analysis or aggregation results based on visualization instructions and/or alerts based on the alert instructions. As measurement data is generated by a measuring device104in response to a configuration request, the corresponding measurement data is accessed from the measurement database222(via the first API204), analyzed, aggregated, and prepared for visualization based on the analysis instructions and visualization instructions included in the configuration request.

The analysis instructions can specify, for example, particular fields of the measurement data to process and mathematical and/or statistical algorithms to use for processing the measurement data. The visualization instructions can specify, for example, which measurement data to display, how to display them, which graphical characteristics to use (e.g., graph types, layout, color), and which numerical statistics to display (e.g., max, min, mean).

The analysis module210further detects alert conditions associated with measurement data and/or analysis results determined from the measurement data using alert instructions specified in the corresponding configuration request. When an alert condition is detected, a visual indicator can be included with corresponding visualization data so that the client device108will display an alert indicator. Alternatively, the alert indicator can output to the client device108as an audio or vibratory indicator that will cause the client device104to generate an auditory or vibratory alert.

The aggregation module212aggregates measurement data generated from tests performed by the measuring devices104. Aggregation can be performed, for example, based on users, attributes associated with the configuration requests, measuring devices104used, etc. The user configuration interface202can provide a GUI that indicates aggregation results available. The aggregation results can be updated periodically or in real time to incorporate measurement data that is received from measuring devices104. Users can select from available aggregation results or generate aggregation requests to aggregate measurement data. The aggregation requests can include aggregation instructions and visualization instructions.

The aggregation instructions can specify attributes of measurement data and results of processing performed on measurement data to include in the aggregation and analysis operations to perform on the data included in the aggregation. Inclusion of measurement data can be based on attributes associated with configuration requests used to perform tests that generated the measurement data, attributes associated with the test, and/or attributes associated with measurement data. Examples of configuration request attributes include title, icon, picture, description, author, date entered, tag, and label. Examples of test attributes include date/time performed, measuring devices104used, location performed, and user that activated the test. Examples of measurement data attributes include identification of the associated measuring device104, attribute(s) of the measuring device104(e.g., manufacturer, property it measures, etc.), location of the measuring device104, date/time stamps associated with the measurement data, and measurement data content. Authors and users can be specified based on attributes, such as profession, employer, country/place of residence, gender, race, age, etc.

The aggregation request can also specify a comparison operation in which a first and second set of data are compared and the results are visualized. The first and second sets of data can be measurement data or aggregation data and can be selected based on attributes associated with the measurement data or aggregation data and/or the configuration requests or aggregation requests used to generate the data sets.

The aggregation visualization instructions can provide instructions to the UI interface202for generating visualization data for displaying aggregation results. The aggregation visualization instructions can specify, for example, which aggregation results to display and how to display the aggregation results, such as layout, which graphical characteristics to use (e.g., graph types, layout, color), and which numerical statistics to display (e.g., max, min, mean).

The deployment module214can deploy a measuring device104in accordance with received deployment instructions. The deployment module214can receive from the user configuration interface notification and identification of a received configuration request that includes deployment instructions and accesses the associated deployment instructions from the configuration database220. Alternatively, the configuration module206can receive the deployment instructions from the user configuration interface202.

The deployment instructions can specify where (e.g., a location on a network230being monitored) to deploy a virtual measuring device104. The deployment module216can deploy a measuring device104on network108based on the deployment instructions.

The analysis module210, aggregation module212, and deployment module214can interact with the storage system106to access identified configuration and/or aggregation requests from the configuration database220, store, access, update, and retrieve measurement data from the measurement database222, and store, access, update, and retrieve aggregation results from the aggregation database224.

Accordingly, the disclosure provides a system and method for an individual to configure a measuring device104to perform a test; perform customized pre-analysis, storage, and analysis of measurement data; perform aggregation of measurement data; and provide customized visualization of the results. The user configuration interface202can allow a user to upload to the configuration database220and the measuring device104a configuration request that was generated or obtained by the user independent of the measurement configuration server102. In addition, the user configuration interface202can provide a GUI to the user to browse through different configurations requests and instructions (e.g., configuration, deployment, pre-analysis, analysis, visualization, and/or summary instructions) to select and/or compose a configuration request. Once selected, the user can modify the configuration request. Users can upload modified or new configuration requests to the configuration database220for use by others or for use by only a specified one or more users.

Users can specify attributes to associate with a configuration request they added to the configuration database220, such as a title, icon, picture, description, author, date entered, tag, or label. The attributes can be associated to the configuration request as, for example, metadata. These attributes can be used to search, sort, and/or affect display of entries in the configuration database220.

In addition, users can browse, search, and select measurement data results stored in the measurement database222and aggregation requests and aggregation results stored in the aggregation database224. Users can modify selected aggregation requests or generate new aggregation requests, and store these to the aggregation database224. The user configuration interface202can provide a GUI to the user to browse through the measurement database222and the aggregation database224to select measurement data, aggregation requests, or aggregation results and/or compose an aggregation request. Once selected, the user can modify an aggregation request. Users can upload modified or new aggregation requests to the aggregation database224for use by others or for use by only a specified one or more users.

Users can specify attributes to associate with an aggregation request they added to the configuration database220, such as a title, icon, picture, description, author, date entered, tag, or label. The attributes can be associated to the aggregation request as, for example, metadata. These attributes can be used to search, sort, and/or affect display of entries in the aggregation database224.

FIG. 3shows an example screenshot300of a GUI provided to a client device108for browsing the configuration database220. The view shown in the screenshot300shows a palate of icons302associated with network performance testing. Each icon302represents a category of network performance tests to perform. In response to a user selecting an icon302, a menu of available configuration requests and associated instructions can be provided, from which the user can select or use to compose a configuration request. The user can identify one or more measuring devices104to which the configuration request is to be applied, and activate a test of the measuring device(s)104in accordance with the selected configuration request by activating (e.g., “clicking”) a link associated with the configuration request.

The user configuration interface202can provide a GUI that allows a user to search the configuration database220, display entries, or hide entries using attributes, such as titles, authors, date entered, tags, and/or labels. The GUI can further allow users to rank (e.g., “like”, “don't like”) entries in the configuration database220. The measurement configuration server102can track usage and ranking of entries in the configuration database220and the UI220can display in the GUI tracking results of the usage and ranking, such as to indicate popularity. Users may be required to subscribe to the measurement configuration server102to access the configuration database220. Subscribers can be provided with personal accounts to store lists of frequently used entries (“favorites”), track (“follow”) usage by other subscribers, and share information with other subscribers.

With reference now toFIG. 4, shown is a flowchart demonstrating implementation of the various exemplary embodiments. It is noted that the order of operations shown inFIG. 4is not required, so in principle, the various operations may be performed out of the illustrated order. Also certain operations may be skipped, different operations may be added or substituted, or selected operations or groups of operations may be performed in a separate application following the embodiments described herein.

At operation402, a request is received. At operation404, a determination is made whether the request is a configuration request or an aggregation request. If the determination at operation404is that the request is a configuration request, at operation406the configuration module206is notified regarding the configuration request. The configuration request can identify one or more measuring devices104and specify configuration criteria for configuring the measuring device104to perform a test. The configuration criteria can include deployment instructions, pre-analysis instructions, analysis instructions, visualization instructions, and/or aggregation instructions. The configuration request can include the instructions or can identify a stored configuration request that can be accessed from the configuration database220by the user configuration interface202or the configuration module206.

At operation408, a determination is made whether the configuration request includes deployment instructions. If the determination at operation408is YES, then at operation410, the measuring device104is deployed at a location of a computer network130specified by the deployment instructions.

The method continues at operation412. If the determination at operation404is NO, the method also continues at operation412. At operation412, the measuring device104is configured to perform measurements and output measurement data in accordance with the configuration instructions. For example, the configuration instructions may specify a characteristic of information transmitted via a network being monitored, and the measuring device104may perform measurements upon detection of the characteristic.

At operation414, it is determined whether the configuration request includes pre-analysis instructions. If the determination at operation414is YES, at operation416pre-analysis processing is performed on measurement data generated by the measuring device104, and at operation418the results of the pre-analysis are stored in the measurement database222. If the determination at operation414is NO, at operation420the measurement data output from the measuring device104is stored in the measurement database122without pre-analysis, e.g., as raw data.

At operation422, the analysis module210performs analysis processing on the measurement data or pre-analysis results associated with the configuration request. Analysis results are stored in the aggregation database224and visualization data results are provided to the user visualization interface204. The analysis module210can be notified when to perform analysis, for example by the configuration module206or the measurement database212. The analysis can be performed upon detection or notification of a condition, such as upon completion of a test, upon updating the measurement database222, or at regular intervals once measurement data or pre-analysis results are stored in the measurement database222, etc.

At operation424, a determination is made whether an alert condition was detected during the analysis. This determination can be made during and/or after the analysis for one or more alert conditions. If the determination at operation424is YES, then at operation426an alert notification is output to the user visualization interface204. The method continues at operation332. If the determination at operation404is that the request is an aggregation request, at operation428the aggregation module212is notified regarding the aggregation request. The aggregation request can specify aggregation criteria, such as data attributes to use for aggregating measurement data and analysis results. The aggregation request can include the aggregation criteria or can identify a stored aggregation request that can be accessed from the configuration database220by the user configuration interface202or the aggregation module212.

At operation430, the aggregation module212aggregates measurement data and/or result data stored in the measurement database222based on the aggregation criteria and outputs corresponding visualization data that can be used to display the results of the aggregation. At operation332the user visualization interface204provides the visualization data to the client device108so that the client device108can display the visualization data.

Embodiments of measurement configuration server102may be implemented by one or more computer systems. One such computer system, measurement configuration server102is illustrated inFIG. 5. In various embodiments, measurement configuration server102may be a server, a mainframe computer system, a workstation, a network computer, a desktop computer, a laptop, or the like.

Measurement configuration server102is only one example of a suitable system and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the disclosure described herein. Regardless, measurement configuration server102is capable of being implemented and/or performing any of the functionality set forth hereinabove.

Measurement configuration server102is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with measurement configuration server102include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed data processing environments that include any of the above systems or devices, and the like.

Measurement configuration server102may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Measurement configuration server102may be practiced in distributed data processing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed data processing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

Measurement configuration server102is shown inFIG. 5in the form of a general-purpose computing device. The components of measurement configuration server102may include, but are not limited to, one or more processors or processing units516, a system memory528, and a bus518that couples various system components including system memory528to processor516.

Measurement configuration server102typically includes a variety of computer system readable media. Such media may be any available media that is accessible by measurement configuration server102, and it includes both volatile and non-volatile media, removable and non-removable media.

Program/utility540, having a set (at least one) of program modules515, such as user configuration interface202, user visualization interface204, configuration module206, analysis module210, aggregation module212, and deployment module214may be stored in memory528by way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment. Program modules515generally carry out the functions and/or methodologies of embodiments of the disclosure as described herein.

Measurement configuration server102may also communicate with one or more external devices514such as a keyboard, a pointing device, a display524, etc.; one or more devices that enable a user to interact with measurement configuration server102; and/or any devices (e.g., network card, modem, etc.) that enable measurement configuration server102to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces522. Still yet, measurement configuration server102can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via network adapter520. As depicted, network adapter520communicates with the other components of measurement configuration server102via bus518. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with measurement configuration server102. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.