Interactive portal for facilitating the representation and exploration of complexity

Embodiments described herein pertain to a standardized set of tools for representing and exploring the components and characteristics of complex systems. In one embodiment, the tools are deployed in a computer network environment so as to engage a social network such that its members utilize the tools to collaboratively construct and maintain a dynamically evolving learning environment in which complexity is represented and explored.

BACKGROUND

It is difficult to precisely define complexity. Though certainly an oversimplification, one possible view of the concept is that to be complex is to be intricate or complicated. In practice, when complexity is studied, it is often within the context of a system. Complex systems are typically rich with opportunities for discovery and analysis. This is true at least because such systems often include an elaborate arrangement of interconnected parts that are influenced by multiple variables such that certain groups of parts collectively demonstrate properties that are not necessarily apparent in the properties of the individual parts. Unfortunately, some of the most interesting of the more intricate and complicated patterns often go undetected. Or, if detected, such patterns are often not understood well enough to provide meaningful insight. This is unfortunate because such patterns within complex systems have the potential to fuel the discovery of particularly useful and interesting knowledge.

More of these complex patterns would be detected and understood if there were an effective and at least somewhat standardized tool for documenting and navigating through the components and characteristics of a complex system. Further, deploying such a tool in a manner that effectively leverages modern social and computer networking technologies would enable a collaborative and dynamic learning environment that would be especially conducive to the derivation and discovery of complex patterns and therefore the derivation and discovery of useful and interesting knowledge.

The discussion above is merely provided for general background information and is not intended for use as an aid in determining the scope of the claimed subject matter.

SUMMARY

A standardized set of tools for representing and exploring the components and characteristics of complex systems is described herein. In one embodiment, the tools are deployed in a computer network environment so as to engage a social network such that its members utilize the tools to collaboratively construct and maintain a dynamically evolving learning environment in which complexity is represented and explored.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

1. Illustrative Operating Environments

Embodiments of the present invention may illustratively be applied in the context of a computer system.FIG. 1is a block diagram of an exemplary computer system within which embodiments of the present invention may be implemented. The system includes a conventional personal computer10, including a processing unit12, a system memory14and a system bus16that couples the system memory14to the processing unit12. System memory14includes read only memory (ROM)18and random access memory (RAM)20. A basic input/output system22(BIOS), containing the basic routines that help to transfer information between elements within the personal computer10, is stored in ROM18.

Embodiments of the present invention may be applied in the context of computer systems other than theFIG. 1personal computer10. Other appropriate computer systems include handheld devices, tablet devices, multi-touch input devices, e-reader devices, any other consumer electronic device, mainframe computers, multi-processor systems, and the like. Those skilled in the art will also appreciate that embodiments can also be applied within computer systems wherein tasks are performed by remote processing devices that are linked through a communications network (e.g., communication utilizing Internet or web-based software systems). For example, program modules may be located in either local or remote memory storage devices or simultaneously in both local and remote memory storage devices. Similarly, any storage of data associated with embodiments of the present invention may be accomplished utilizing either local or remote storage devices, or simultaneously utilizing both local and remote storage devices.

Personal computer10further includes a hard disc drive24, a magnetic disc drive26, and an optical disc drive30. Magnetic disc drive26can illustratively be utilized to read from or to write to a removable disc28. Optical disc drive30can illustratively be utilized for reading data from (or writing data to) optical media, such as a CD-ROM disc32. The hard disc drive24, magnetic disc drive26and optical disc drive30are connected to the system bus16by a hard disc drive interface32, a magnetic disc drive interface34, and an optical disc drive interface36, respectively. The drives and their associated computer-readable media provide nonvolatile storage for the personal computer10. Other types of media that are readable by a computer may also be used in the exemplary operation environment.

A number of program modules may be stored in the drives and RAM20, including an operating system38, one or more application programs40, other program modules42and program data44. In particular, one or more of the other program modules42can illustratively be part of an interactive portal for facilitating the representation and exploration of complexity. For example, the portal may include features consistent with one or more portal features that are part of embodiments of the present invention described below in greater detail with reference to other Figures.

Input devices including a keyboard63and a mouse65are functionally connected to system bus16through a serial port interface46that is coupled to system bus16. Input devices could be connected utilizing other interfaces such as a universal serial bus (USB). Monitor48is connected to the system bus16through a video adapter50. Other peripheral output devices (e.g., speakers or printers) could also be included but have not been illustrated.

The personal computer10may operate in a network environment utilizing connections to one or more remote computers, such as a remote computer52. The remote computer52may be a server, a router, a peer device, or other common network node. Remote computer52may include many or all of the features and elements described in relation to personal computer10, although only a memory storage device54has been illustrated inFIG. 1. The network connections depicted inFIG. 1include a local area network (LAN)56and a wide area network (WAN)58. Such network environments are commonplace in the art. The personal computer10is connected to the LAN56through a network interface60. The personal computer10is also connected to WAN58and includes a modem62for establishing communications over the WAN58. The modem62, which may be internal or external, is connected to the system bus16via the serial port interface46. These are but examples of network implementations within the scope of the present invention.

In a networked environment, program modules depicted relative to the personal computer10, or portions thereof, may be stored in the remote memory storage device54. For example, application programs64may be stored utilizing memory storage device54and may include at least a portion of an interactive portal for facilitating the representation and exploration of complexity, embodiments of which will described below in greater detail below with reference to other Figures. In addition, data associated with an application program, such as the interactive portal consistent with embodiments of the present invention, may illustratively be stored within data storage66. Data storage66is associated with memory storage device54(alternatively, or in combination, the application data may be stored within one of the data storage locations associated with computer10). It will be appreciated that the network connections shown inFIG. 1are exemplary and other means for establishing a communications link between the computers, such as a wireless interface communications link, may be used.

Notwithstanding the broad applicability of the principles of the invention described above, it should be understood that the configuration of the exemplary embodiment as an application program that is stored on, and enables data to be stored on, a remote computer such as remote computer52provides significant advantages. For example, such a configuration minimizes the functional requirements (e.g., storage capacity) of personal computer(s)10and enables a broader range of potential adaptive control features, simplified system updating and upgrading features, and also simplified debugging and troubleshooting features.

2. Illustrative Application Architecture

FIG. 2is a schematic representation of a complex information interaction system100that encompasses an interactive portal for facilitating the representation and exploration of complexity, various components and features of which will be described throughout the remainder of the present description. System100includes server components102, which include one or more databases106. Databases106are illustratively utilized within system100to store (in whole or in part) representations162of complex systems and a related framework164, embodiments of which will be discussed below in relation to other Figures. The server components102also include a complex information interaction application104and a set of data management components108. Data management components108are illustratively configured to respond to input commands (e.g., automatically generated and/or human-user initiated commands) originating from the application104. For example, components108illustratively interact with database106so as to manage the storing and retrieving of the representations162, framework164, and other related data based on commands from the application104. Components108are also configured to manage any of a variety of other database management functions such as, but not limited to, data access management and functions related to maintaining data security.

Those skilled in the art will appreciate that application104, data management components108and database106need not all be, as illustrated, in the same server environment or even in the same location. They could just as easily be distributed across multiple server environments and/or multiple locations without departing from the scope of the present invention. All possible combinations of distribution are contemplated and are to be considered within the scope of the present invention.

Complex information interaction application104is illustratively a network-based application that includes a public web interface maintained on the Internet or some other computer network. In one embodiment, application104is coded in a browser-supported language. For example, not by limitation, application104can be coded in JavaScript in combination with a browser-rendered markup language such as, but not limited to, HTML. The code might also encompass style sheets as a means for influencing the presentation of the application to the user. Those skilled in the art will appreciate that this is but one of many examples of how a web-based application can be implemented without departing from the scope of the present invention. Other implementations, including any coding combination implementing any web application development tools are also within the scope. Regardless of the precise implementation, users are illustratively able to interact with application104remotely utilizing any of a wide variety of different computing devices. Devices within the scope of the present invention include, but are certainly not limited to, a mobile phone, a laptop PC, a desktop PC, a mobile device, a personal digital assistant, a multi-touch input device, an e-reader device, a tablet PC, a portable music player, or any other similar electronic device.

Accordingly, the user-focused front end of application104is, in one embodiment, reliant upon a client side web browser for ultimate execution, including presentation to and interaction with a user of the browser/application. For this reason, browser applications107,109,105and111are shown inFIG. 2as supporting access to application104by users110,112,114and120, respectively. As will become apparent, users110,112and114(and probably also the gatekeeper user120) are able to utilize application104in combination with their browser application to experience (e.g., view, listen to, navigate through, etc.) the complex system representations162(and the related framework164) stored, in whole or in part, in database106. Further, users110,112and114(and probably also the gatekeeper user120) can also, if they wish, utilize application104in combination with their browser application to make contributions to and edit the complex system representations162(and related contextual portions of framework164) stored, in whole or in part, in database106. As will also become apparent, gatekeeper120manages, to some extent, modifications and additions to the representations162(and related contextual portions of framework164) stored, in whole or in part, in database106.

For the purpose of simplifying the description of the present invention, only the representative set of users110,112,114and120are shown as remotely accessing application104over a network103, which is illustratively but not necessarily the Internet. Further, it should be understood that it is contemplated that different users could, in theory, access application104over different computer networks rather than, as is illustrated inFIG. 2, over the same network. Further, one or more users could access application104by means other than over a public or other computer network. For example, user120is shown inFIG. 2as having both network-based and more direct access to application104. User120may, in one embodiment, also have network-based or more direct access to either or both of components106and108. Any combination of computer-implemented user access connection/communication schemes from accessing the server components102should be considered within the scope of the present invention. Any user can be provided with public remote and/or more direct access to the server-based components of system100(e.g., components102) without departing from the scope of the present invention.

Gatekeeper120is illustratively a system administrator having broader access rights and privileges than users110,112and114. Gatekeeper120controls, at least to some degree, the configuration and functionality of components104,106and108, and therefore has the ability to affect the application104experience from the perspective of users110,112and114. In one embodiment, the gatekeeper120sets and/or enforces conditions under which the complex system representations162and framework164may be modified or expanded upon by users110,112and114. The described network architecture enables gatekeeper120to make system adjustments and modifications without having to create and distribute frequent software updates to the users. Instead, the gatekeeper is able to make changes quickly and, if desired, frequently. Of course, it is contemplated that there may be multiple or even many gatekeepers working independently or cooperatively to maintain and manage the database106content and/or its related functional components.

Those skilled will appreciate that there are many specific implementation options for the physical and functional configuration of database106and its related components108. Each option has its own set of drawbacks and benefits. The best option will depend at least upon available resources both in terms of equipment and personnel. For the purpose of providing a complete and thorough description of embodiments of the present invention, it is worth expounding upon a few different of the many possibilities.

In one embodiment, at least part of database106is implemented as a relational database comprising a collection of tables that store particular sets of data pertinent to the stored representations162, framework164and other related data. However, without departing from the scope of the present invention, database106can include any other type of database technology such as but certainly not limited to an object database, an object-relational database, or a native xml database. All of these and any other database implementation should be considered within the scope of the present invention.

Those skilled in the art will appreciate that the configuration and functionality of components108will vary as necessary to accommodate whichever type of database technology is included as part of the implementation of database106. For example, in a scenario in which database106includes a SQL database implementation, such as MySQL or SQL Server offered by Microsoft Corporation of Redmond, Wash., then the data management components108are likely to include a PHP or Python access/security layer. In an object-relational (O-R) database scenario, components108illustratively include support for referring to stored data in terms of objects and their properties rather than the more specific elements of a more traditional database schema (e.g., a table-based schema). For example, in order to support the O-R system, components108are likely to include a mapping mechanism (e.g., a class-table mapping mechanism) that facilitates a translation of object-oriented database calls into corresponding relational database calls.

In still another embodiment, database106includes an implementation of NoSQL, such as a MongoDB database implementation. In this case, the database will generally not be organized in tables similar to a relational database but instead will be organized around flat stack objects. One advantage of a NoSQL implementation in the context of the present invention is that it allows for especially efficient access to what are likely to be particularly large data sets. This is true because, generally speaking, a NoSQL implementation requires fewer server resources than, for example, a relational database. Of course, to accommodate a NoSQL implementation, appropriate data management components108will be cooperatively implemented. For example, when database106includes a NoSQL implementation, components108illustratively include a Python, or Ruby, or PHP access/security layer. Of course, those skilled in the art will appreciate that these are but examples of possibilities within the scope of the present invention.

Those skilled in the art will also appreciate that components106/108are configured to support any of a plurality of different data format and exchange protocols that enable communication between application104and the remote computers of the users110,112,114and120. In one embodiment, the protocol is such that it supports a pulling of live code from the database for immediate execution in conjunction with application104and/or in conjunction with an application (e.g., the browser application) operating on one of the remote user computers. This capability is particularly well suited for data generative functions of application104contemplated in accordance with embodiments of the present invention that will be described below in greater detail in relation to other Figures.

One option for the data format and exchange protocol is XML. However, in one embodiment, application104is an Ajax-style web application that supports out-of-band, lightweight calls to database106. These calls are illustratively initiated using JavaScript and involve formatting data, sending it to the web server, and parsing and working with the returned data. While most browsers can construct, send, and parse XML, JavaScript Object Notation (or JSON) provides a standardized data exchange format that is better-suited for Ajax-style web applications. For example, the parsing required in an XML implementation is simplified or eliminated in the context of a JSON implementation. Thus, in one embodiment, but not by limitation, JSON is utilized as the data format and exchange protocol at least in, but not limited to, embodiments where the application104involves a JavaScript implementation in the context of an Ajax web application.

In terms of the particular configuration of components108and database106, the scope of the present invention is not limited to any one particular setup in conjunction with a JSON implementation. For example, in one embodiment, database106includes a MongoDB implementation in conjunctions with a Python or Ruby (or any other suitable implementation) in the access/security layer. However, another option within the scope of the present invention is a SQL-type database106implementation in conjunction with a PHP access/security layer configured to facilitate a conversion into JSON-type stacks. These are but two examples of JSON implementations within the scope of the present invention. The scope is not limited to these or any other particular implementation.

In accordance with one embodiment of the present invention, at least one of the representations162(and its associated portions of framework164) accessed by way of application104is graphically presented (e.g., presented in the context of a portion of framework164) within the user's browser based on an implementation of Web-based Graphics Language (WebGL) technology. Those skilled in the art will appreciate that WebGL generally refers to a technology involving utilization of JavaScript so as to implement 3D graphics within a user's browser. More particularly, WebGL is a context of the canvas HTML element that provides a 3D graphics API implemented in a web browser without the use of a plug-in. Of course, the WebGL functionality requires the user to operate a WebGL compatible browser. However, it is believed by many in the art that most browsers in the future will support WebGL graphics technology. Compatible browser applications are presently readily available on the Internet and otherwise. It should be noted that the scope of the present invention is not limited to WebGL technology specifically. Other similar technologies (e.g., similar technology offered by Microsoft Corporation of Redmond, Wash.) can be included in the system integration without departing from the scope of the present invention.

Audio within web applications is currently most frequently delivered through plugins such as Flash and QuickTime. This may be true in the context of system100. However, in accordance with one aspect of the present invention, at least one of the representations162accessed by way of application104includes audio presented within the user's browser based on an implementation of a JavaScript API for processing and synthesizing audio. Accordingly, in one embodiment, audio is synthesized and processed directly in JavaScript without use of a plug-in. In one embodiment, audio is managed through the AudioData API (Mozila Firefix) and/or the WebAudio API (Chrome and Sarari) thereby enabling audio information to be efficiently manipulated directly or at least relatively more directly upon a user's hardware responsible for audio data management. In one embodiment, the audio is implemented within the applications such that the representation includes the audio used with graphics generated using the canvas 2D and/or WebGL 3D graphics APIs.

3. The VisualY Language

Words that humans think, speak and write are mostly based upon the incremental development of language instead of any purposefully invented system of communication. In most cases, the origin of these words dates back hundreds and even thousands of years—perhaps 50 to 100 generations. However, it has been only within about the past 400 years (spanning just 10 or so generations) that an understanding of the science of our world and existence entered a degree of accuracy and certainty that could reasonably serve as a workable outward foundation of accurate knowledge and thought.

Under the circumstances, language has developed into something other than a scientifically founded invention having as its objective being a most effective means of organizing and communicating information and knowledge. Instead, language has become a significantly limited and generally imprecise accumulation of human constructs that once established became very difficult to modify or discard. And because humans generally think within the confines of language, they therefore find it very difficult to work outside of it. It is therefore both reasonable and logical for the continued advancement of human understanding that current language-driven systems for organizing and communicating information and knowledge be improved or replaced.

Application104, in accordance with one aspect of the present invention, is implemented in a manner the same (or similar) as described in relation toFIG. 2but is done so as part of a scheme for overcoming the noted and other shortcomings of current systems of language. More particularly, application104is programmed to provide an advanced computer-implemented system for managing and presenting information and knowledge in a way that more accurately addresses how the components of the world and universe are actually organized—in particular as synchronous and asynchronous complex systems. In contrast, current systems of language are frustratingly linear in that they are largely dependent upon words that in many cases are only capable of expressing a very limited scale and scope of complexity.

In one embodiment, a key function of application104is to facilitate representation and communication of information and knowledge in accordance with a dynamic interactive visual and auditory language, which will be referred to herein as the language of VisualY. As will become apparent, an instantiation of the VisualY language will most often be presented as a visual and/or auditory representation of information/knowledge set against the context of all or a portion of a multi-level ontological system of maps and map views. The system of map and map views is illustratively the framework164inFIG. 2. The framework164may be referred to herein as the multi-level ontological model framework164, as model framework164, or as the VisualY model framework164. The information/knowledge representations are illustratively the representations162inFIG. 2. The representations162may be referred to herein as the VisualY representations162.

Application104utilizes a management component166to facilitate the creation, organization, collection, modification, transformation, presentation, navigation, browsing, searching, experiencing, displaying, storage, retrieving, transference, and otherwise communication or manipulation of instantiations of the VisualY language. Accordingly, application104utilizes management component166to facilitate the creation, organization, collection, modification, transformation, presentation, navigation, browsing, searching, experiencing, displaying, storage, retrieving, transference, and otherwise communication or manipulation of the VisualY representations162, most often presented in the context of all or a portion of the VisualY model framework164. While the representations162and the model framework164are being presented as elements separately stored in database106, the line between these two components in actual implementation is not likely to be so distinct. Accordingly, it is likely also true that application104utilizes management component166to facilitate the creation, organization, collection, modification, transformation, presentation, navigation, browsing, searching, experiencing, displaying, storage, retrieving, transference, and otherwise communication or manipulation of the VisualY model framework164.

Application104also includes a set of user-driven development tools168. Tools168, which will illustratively be referred to herein as the VisualY tools or collectively as the VisualY toolset, are utilized by a user (e.g., by any or all of users110,112,114and120interacting with application104by way of a web browser) to interact with instantiations of the VisualY language as described in the paragraph before this one. This means that the VisualY toolset168may be utilized by the users to create and/or modify VisualY representations162and/or the VisualY model framework164. In one embodiment, it is within the scope of the present invention for each user to each create their own VisualY instantiations and utilize functions included within application104to share access and/or modification rights as desired with one another. However, in another embodiment, the application is configured to enable all of the users to access and/or modify the same VisualY instantiations.

Thus, in one embodiment, application104operates in a manner similar to the known Wikipedia system currently made available on the WorldWideWeb at www.wikipedia.org. For example, application104illustratively enables the users to selectively access VisualY instantiations but the users are also able to use the VisualY toolkit168to collectively, as the community of users of application104, add to, delete or otherwise modify the VisualY instantiations or at least portions thereof. This illustratively includes the collaborative adding to, deleting and/or modifying of the VisualY model framework164and/or the VisualY representations162.

In one embodiment, one or more gatekeepers, an example of which is gatekeeper120, are provided with access to administrative functions that enable them to approve or reject changes made by the broader user community to one or more instantiations of the VisualY language. Those skilled in the art will appreciate that this authority to accept or reject changes can be manifested in many different ways. For example, in one embodiment, no changes are allowed without approval by a gatekeeper. In other embodiment, changes made by some users are allowed without approval but changes made by other users require approval. Further, different users may be assigned different access rights in terms of what specific types of changes they are allowed to make to VisualY instantiations. These rights may vary from one instantiation to the next or may be consistent across an entire collection of instantiations.

In response to user-initiated command interactions with application104, a VisualY instantiation (e.g., a combination of a representation162and a corresponding model framework164context) is retrieved by component166and, in one embodiment, all or a portion of it is presented to a user (e.g., displayed on a display such as but not limited to display48inFIG. 1, and/or provided in part as audio, is provided in a conveniently printable format, etc.). In one embodiment, this process includes data management components108receiving from component166a request (e.g., a user or application initiated query originating from one of browser applications105,107,109or113) that specifies, either directly or indirectly, all or a portion of the VisualY instantiation to be retrieved from database106. The requested representation/model framework data is retrieved from the database106and returned to components108in the form of database results. In one embodiment, components108include a data formatting component that formulates the database results into a proper format (e.g., a format identified in the request, identified within the retrieved data or otherwise identified in the code of application104), such as in the form of one or more objects or simply as a data set. In any case, the properly formatted data is provided as an output to application104in response to the application request. The results are transmitted over network103and presented to the user (e.g., the requesting user) (e.g., one or more of users110,112,114or120) through their browser application, which facilitates execution of software code that is part of application104, though it is within the scope for executed application code to originate from other sources such as but not limited to being embedded within data stored in the database106.

FIG. 3is a diagrammatic representation of a screenshot300that includes one embodiment of a portion of the VisualY model framework164(i.e., the multi-level ontological model framework164). The portion shown inFIG. 3represents but one level of the model framework. Line306divides the illustrated level into left section308and right section310. Each of sections308and310is divided into a collection of cells312, only an exemplary set of which have been included in order to simplify the illustration. Situated at the bottom of the screenshot is a plurality of buttons314-322that are not technically part of the model framework but instead are controls for performing functions in relation to a representation162depicted in relation to a portion of a displayed model framework level, or for transitioning between representations162and/or portions of the model framework164.

Before proceeding with the description of screenshot300and its illustrated portion of the model framework164, it is first worth noting that, without departing from the scope of the present invention, the configuration and shape of cells312may be very different than what is shown in any Figure included herewith. For example, straight lines might be replaced with non-straight lines, or there could be more or fewer cells312, etc. The stylistic design choices reflected in the Figures provided herewith are intended to be exemplary only and are in no way to be construed as limiting the scope of the present invention.

In one embodiment, some or all of cells312are user-selectable (e.g., but not by limitation, by way of a mouse click, voice recognition, touch screen input, gesture recognition, etc.) so as to cause navigation (e.g., a transition of the display/screenshot) to a different level of the model framework164. When a symbol appears within a cell312, the symbol illustratively is representative of the subject matter and/or configurations of other cells312obtainable through the symbol-containing cell. For example, in one embodiment, not by limitation, the cell312that includes symbol324indicates to the user that there are at least four additional cells312that can be navigated to by selecting the cell312(i.e., one symbol for each of the additional cells312). Similarly, the cell312that includes symbol326indicates to the user that there are at least two additional cells312(i.e., represented by the two symbols) that can be navigated to by selecting the cell312that encompasses the two symbols. This is not to say that there must always be a symbol for every cell312reachable through the symbol-containing cell312. The numbers of symbols included in a given cell312, for example, may be dependent upon the current displayed size of the given cell. In one embodiment, the application is configured to change the size of the symbols or even change the symbols themselves depending upon cell size. In one embodiment, a short-hand symbol (e.g., a dot) is utilized to show existence of a particular symbol when the symbol-containing cell is not large enough to display the particular symbol in a visually appealing way.

It should be noted that the described symbol-based scheme is only an example of how selectability/navigability might be indicated. For example, but not by limitation, the indication could just as easily be a more unified symbol such as a shape containing a dot for each cell312in the next selectable level of the model framework. Further, the indication need not necessarily include any symbol per se. It is within the scope of the present invention for the indication to be a letter, a number, a plus sign, any other character, a shape that is not a letter or number, a color, cell shape, cell location, cell location in relation to a separate indication of selectability, a sound, a change in color, a change in sound, a change in characters, or even an animation. Any of these or any other possibility can be utilized to indicate that a particular cell312is selectable/navigable to a different level of the model framework164.

It is also to be understood that an indication of selectability/navigability may be selectively displayed such as based on user preferences or interactions. For example, in one embodiment, an indication is only activated and provided when the user has expressed an interest in the associated cell312(e.g., for example, but not by limitation, by “mousing over” the cell). Any process for temporarily or conditionally presenting one or more of the indications should be considered within the scope of the present invention.

The symbols included within a cell312may or may not be an indication or reminder of a type of subject matter pertinent to the cell and/or pertinent to corresponding lower levels of the model framework164accessible via selection/navigation of the cell. For example, a plant symbol might indicate that components of the model framework164accessed through the cell containing the plant symbol pertain to plant-oriented subject matter. It should be noted that an indication of subject matter relevancy need not necessarily be a symbol per se. It is within the scope of the present invention for the indication to be a letter, a number, any other character, a color, cell shape, cell location, a sound, a change in color, a change in sound, a change in characters, or even an animation. Any of these or any other possibility can be utilized to indicate that a given cell312pertains to a particular category of subject matter. It is also to be understood that the subject matter indication need not always be displayed for a given cell. In one embodiment, the indication is only displayed when the user has expressed an interest in the associated cell312(e.g., for example, but not by limitation, by “mousing over” the cell). Any process for temporarily or conditionally displaying one or more of the subject matter indications should be considered within the scope of the present invention.

As has been described, the screenshot ofFIG. 3represents only a portion of the VisualY framework164, namely one level of the framework.FIG. 4is a diagrammatic representation of a screenshot400displaying another level included in the framework. The level shown inFIG. 4includes its own collection of cells312. The plurality of buttons314-322are again situated at the bottom of the screenshot inFIG. 4. Again, these buttons are generally not part of the model framework but instead are controls for performing functions in relation to a currently presented VisualY instantiation.

TheFIG. 4screenshot is illustratively what the user sees upon selection of the cell312inFIG. 4that includes symbol324. In other words, the user navigates from the level of the model framework shown inFIG. 3to the level shown inFIG. 4by selecting the cell312containing symbol324. Notably, the cell in theFIG. 3screenshot that includes symbol324is one of four symbols in the cell. The same four symbols appear again in theFIG. 4screenshot but this time as labels for a set of different cells312that include new symbols. The new cells in theFIG. 4screen shot can illustratively be selected in order to navigate to still another level of the model framework164. A similar but different set of symbols is utilized in theFIG. 4cells312to again indicate navigability options and/or subject matter relevancy.

FIG. 5is a broader schematic illustration of an embodiment of a multi-level ontological model framework164, illustratively the same framework reflected in the screenshots ofFIGS. 3 and 4. This schematic illustration is intended to make clearer the hierarchically navigable nature of the model framework164. Each of a plurality model segments502(only a representative few have been specifically labeled inFIG. 5) illustratively corresponds to a cell312included in the framework164. For example, the model segments502on a top level506illustratively correspond to the cells312included in the level of the model framework shown in the screenshot ofFIG. 3. The model segment furthest to the left on level506illustratively corresponds to the cell inFIG. 3that includes symbol324. The arrow552inFIG. 5pointing from level506to level508indicates that the user is provided with the ability, if desired, to select the symbol324cell so as to initiate a transition to the four pointed-at model segments502in level508. These four model segments in508illustratively correspond to the cells312shown inFIG. 4such that the transition between levels506and508illustratively includes a transition of the user interface, as was described in relation toFIGS. 3 and 4.

In addition to levels506and508, the illustration ofFIG. 5also includes levels510and512. Each level includes its own model segments502, which illustratively correspond to cells312included in other levels of the framework164, and therefore in other screenshots available for display depending user-initiated transitions between levels. Arrows between levels are utilized to represent opportunities to navigate to a particular sub-set of cells312that correspond to a sub-set of model segments502on a given level.

It is to be noted that the illustration ofFIG. 5does not exactly correspond to the example screenshots shown inFIGS. 3 and 4. The illustration is simplified at least in that an actual implementation very well may include many more model segments502corresponding to many more cells312included within the model framework164. Further, an actual implementation very well may include many more levels than just the four levels shown inFIG. 5. Still further, the navigation paths (e.g., schematically represented inFIG. 5by arrow552and the other similar arrows) very well may be more complicated than a simple one-to-several linking of model segments502. In one embodiment, multiple model segments502are linked to the same group of model segments502indicating that more than one cell312can be selected/navigated to the same screenshot view of a given sub-set of cells312within the model framework164. Those skilled in the art will appreciate the many ways, all to be considered within the scope of the present invention, in which the model framework164is able to evolve so as to become more complicated than the illustrated scenario. The illustrated scenario has been intentionally simplified in order to make clear the broad nature of various embodiments of the present invention. The navigation paths incorporated into an actual implementation of model framework164can skip levels of the hierarchy up or down, or can otherwise be configured in any manner without departing from the scope of the present invention.

In accordance with one aspect of the present invention, the system is configured to support navigation from a first level within the model framework164to a second level even when there is not a cell312in the first level that supports direct navigation to the second level. In other words, the user is provided with options for navigating between sub-sets of cells312without necessarily following a “connected” path between levels of the framework (e.g., levels506,508,510and512).

FIG. 6is a diagrammatic representation of a navigation history tool600. Tool600is an example of one embodiment of how a non-connected navigation path between levels of the framework164can be implemented as part of application104. Component600is illustratively rendered on the application display so as to provide a visual indication of a recent (e.g., most recent) navigation path taken through framework164in order to arrive at a currently displayed subset of cells312. Component600includes four boxes chronologically arranged from left to right such that the currently viewed level corresponds to the box furthest to the right (i.e., box608) and the first viewed level corresponds to the box furthest to the left (i.e., box602). Of course, the two boxes in the middle (i.e., boxes604and606) represent consecutive intermediately viewed levels along the navigation path from the first viewed sub-set of cells312to the currently viewed sub-set of cells312.

The symbols utilized in each of the boxes of component600are illustratively indicative (e.g., indicative of subject matter, etc) of the corresponding sub-set of cells312. For example, the symbol shown in box602reflects the fact that the user began at the sub-set of cells312reflected in the screenshot300ofFIG. 3. The triangular symbol shown in box604reflects the fact that the user selected a cell312that caused navigation to a second sub-set of cells312(not illustrated in the Figures of the present description). The oval-shaped symbol shown in box606reflects the fact that the user selected a cell312that caused navigation to a third sub-set of cells312(not illustrated in the Figures of the present description). Finally, the symbol shown in box608reflects the fact that the user selected a cell312that caused navigation to a currently displayed sub-set of cells312(not illustrated in the Figures of the present description).

Component600is illustratively configured to support navigation from the screenshot view of cells312associated with box608to the screenshot view of cells312associated with box602without visiting the screenshots view of cells312associated with either of boxes604of606. The user is essentially able to “jump back” to the screenshot of box602(i.e., the view ofFIG. 3) by simply selecting (e.g., clicking on, voice activating, touch screen input selecting, gesture detection inputting, etc.) box602from user interface component600. Thus, the user is not forced to back out through the previously visited screenshot views associated with the intermediate boxes604and606to obtain the previously visited view. The user can, of course, assumedly utilize component600to jump back to any screenshot view indicated in the user interface component600regardless of how many previously visited sub-sets of cells312are included in the record of navigation history. This is but one example of how the hierarchy of the model framework304can be navigated through without being strictly bound to a hierarchical sequence.

Another aspect of the present invention pertains to still another embodiment of how application104is configured to support navigation between non-connected collections of cells312.FIG. 7is a diagrammatic representation of a screenshot700. Screenshot700is illustratively an alternative view of portions of the multi-level ontological model framework164. For example, in one embodiment, a user of application104has the option of choosing the broader and more encompassing framework display scheme ofFIG. 7instead of (or, as will become apparent, in combination with) the framework display scheme described in relation toFIGS. 3 and 4. While those skilled in the art will appreciate that there are many user input control options for effectuating a transition between different framework display schemes, in one particular embodiment the transition is initiated at least semi-automatically when the user inputs a particular framework navigation command that triggers the transition even though the user may not entirely expect the transition to be a triggered by the particular command. Alternatively, the transition may be initiated more intentionally, for example, when a user chooses to activate a scheme selection mechanism made accessible by way of one of buttons314-322or otherwise.

The framework display scheme ofFIG. 7differs from the scheme ofFIGS. 3 and 4in that it represents much more, if not all of, the model framework in a single screen view. In accordance with the FIGS.3/4scheme, the user navigates from one cell312collection to another, for example, from a cell cluster on one level to a cell cluster on another level or between two cell clusters on the same level, such that one cell cluster is shown at a time. However, it should be noted that it is certainly within the scope of the present invention for the FIGS.3/4scheme to be adapted to support renderings of multiple cell clusters from different framework levels (or even on the same framework level) to be simultaneously rendered and presented to the user in the same screenshot. For example, the framework and related navigation scheme could just as easily be configured such that screenshot400ofFIG. 4includes multiple separate sets of cells312from the same level or from different levels. It is worth noting that the generally two-dimensional scheme of FIGS.3/4make it difficult to depict relationships between cell clusters, especially between cell clusters from different levels. This is disadvantage because certain VisualY representations162will utilize color, sound, motion or other information representing mechanisms to show, in the context of the framework164, relationships between cell clusters and components of cell clusters.

In contrast, the framework display scheme ofFIG. 7enables a simultaneous presentation of multiple cell312collections from multiple or all levels of the framework (e.g., levels506,508,510and512as shown inFIG. 5). The scheme is even extensible to show multiple cell clusters from the same framework level. In one embodiment, the screenshot700ofFIG. 7is configured such that a user can select (e.g., by way of voice, mouse, touch, gesture or any other input means described herein or otherwise) any of the individual collection of cells312. This type of selection illustratively causes a direct transition to a different, corresponding display configuration. For example, in one embodiment, selecting item708from top level506illustratively causes the display to transition to a framework164view consistent with screenshot300shown inFIG. 3. In another example, selecting item706from level508illustratively causes the display to transition to a framework164view consistent with screenshot400shown inFIG. 4. This is not to say that the scope of the present invention is limited to transitioning to a “flat” depiction of cell clusters similar to the FIGS.3/4presentation scheme. In one embodiment, a cell cluster presentation to which navigation occurs from a component of theFIG. 7representation is also relational in that it includes depictions of other relatively closely related model framework components. In one embodiment, the related components are greyed out or otherwise de-emphasized in order to highlight a particular cell cluster as the cluster to which the user has navigated relative to the components that are only related to the particular cell cluster. Of course, a mechanism for transitioning back to the “bigger picture” framework view ofFIG. 7is likely to be included as a user-selectable option. This could be but is not limited to a mechanism similar to that shown inFIG. 6and/or a mechanism made accessible by way of one of buttons314-322.

In another embodiment, the framework display scheme consistent with screenshot700ofFIG. 7is configured to enable user selection (e.g., by way of voice, mouse, touch, gesture or any other input means described herein or otherwise) of any of the open space710(only some examples of such space are identified inFIG. 7) on a given framework level. This type of selection illustratively causes a direct transition to a corresponding display of multiple or all sub-sets of cells312on the selected level (as compared to screenshot400inFIG. 4, which only shows one sub-set of cells312). Each sub-set of cells312illustratively is configured to behave and enable subsequent navigation in a manner similar to that described in relation to the single sub-set of cells312shown in screenshot400ofFIG. 4. Of course, in one embodiment, the system is also configured to enable a transfer from a display of multiple cell312sub-sets to a display of a single cell312sub-set that looks and behaves similar to the cell312sub-set shown inFIG. 4. Again, a mechanism for transitioning back to a previously displayed framework view is likely to be included as a user-selectable option. This could be, but is not limited to, a mechanism similar to that shown inFIG. 6and/or a mechanism made accessible by way of one of buttons314-322.

It is worth again emphasizing that the framework display scheme of FIGS.3/4can easily be modified to support transitions to a display of multiple sub-sets of cells312. This may include multiple sub-sets of cells312from the same level (e.g., as in the same level from the levels506-512of the schematicFIG. 5). In one embodiment of this functionality, selecting an instance of open space360(only some examples of such space are identified inFIG. 3) causes a direct transition to a corresponding display of multiple or all sub-sets of cells312on the selected level (as compared to screenshot400inFIG. 4, which only shows one sub-set of cells312). Again, a mechanism for transitioning back to a previously displayed framework view is likely to be included as a user-selectable option. This could be but is not limited to a mechanism similar to that shown inFIG. 6and/or a mechanism made accessible by way of one of buttons314-322.

Those skilled in the art will appreciate that application104is configured to change the size and/or presentation of sets of cells312, the cells312themselves and/or depictions of framework levels (e.g., levels506-512) as necessary to efficiently utilize available screenshot space in a visually appealing way. For example, the described transition from a display similar to screenshot700to a corresponding display of multiple or all sub-sets of cells312on a selected level is, in one embodiment, accomplished by terminating the display of the non-selected levels (and associated sub-sets of cells312) and then expanding the displayed size of the selected level (and associated sub-sets of cells312) to more efficiently fill available display space. This is not to say that the non-selected levels need necessarily be removed in their entirety. It is within the scope of the present invention for the non-selected levels (and associated sub-sets of cells312) to be de-emphasized without being removed (e.g., they can be grayed out, be otherwise made inactive, shrunk, moved to a secondary position, etc.). When an indication of the non-selected levels does remain on the display, the selected level may at least be given a more prominent position and/or more prominent display characteristics.

In one embodiment, the display characteristics of a selected level are automatically adjusted so as to make its own included selectable components easier to select in terms of the user dexterity required to accomplish the selection. For example, the size of the touch, click, etc. selection controls area associated with each of the sets of cells312, the individual cells312, and/or any selectable empty space are illustratively increased in size in order to make them more easily actuated. The opposite is true when more framework components are added within a given screen shot—sizing is reduced in order to accommodate more information on the screen at the expense of smaller components becoming more difficult to select (or, in one embodiment, even impossible to select without re-sizing).

Up to this point, the re-sizing and re-configuring of displayed framework164components and related selection controls has been most often described as being automatically effectuated by application104in response to user-initiated navigation transitions between different framework views. In accordance with another aspect of the present invention, the system is configured to support such adjustments in response to reconfiguration commands received from the user. For example, a user is illustratively provided with the ability to use a user input mechanism (e.g., a mouse, touch input, multi-touch input, voice command input, etc.) to communicate commands that cause displayed components of the framework to be rotated, re-sized and/or re-positioned. It is also within the scope of the present invention for the system to automatically or in response to input (e.g., user input) change the perspective of a cell, group of cells, multiple groups of cells, etc. For example, but not by limitation, the depiction of cells312shown inFIG. 4can be modified from the shown two dimensional depiction to a three dimensional representation or a perspective view similar to the perspective utilized to show the various framework levels shown inFIG. 7.

Screenshot700is illustratively configured such that the user can, utilizing a user input device to communicate user input commands, manipulate the depictions of any of the framework levels506-512along with their associated sets of cells312. In one embodiment, any of the level depictions can individually or collectively be increased or decreased in size. For example, this could be accomplished by way of a touch-input gesture involving a spreading apart or moving together of two distinct touches (i.e., moving fingers apart or together in a manner similar to that currently used in the art to re-size photographs, etc.). When the level depiction is re-sized the associated cells312and/or related selection input control mechanisms are also re-sized, as was previously described. Further, any of the level depictions can individually or collectively be rotated (e.g., tipped forward, backward, or side-to-side) in terms of the perspective of their 3D appearance. Still further, the level depictions can be re-arranged, for example, through utilization of a “select-and-drag” functionality, as is known in the art. In one embodiment, the user interfaces include a mechanism that enables the user to return components they have moved and/or resized back to a default configuration for the currently displayed components of the framework.

In another embodiment, the system is configured to make adjustments automatically in response to reconfiguration commands directed by the user at a particular cell312or collection of cells312. Referring toFIG. 3, the display is illustratively configured to allocate an amount of space given to one or more of the displayed cells312in response to a user-initiated input causing a re-sizing of one or more cells312. For example, space on side308is illustratively increased (and likely the size of cells312and associated input control mechanism) when the space on side312is decreased. In other example, a single cell312that is re-sized by a user causes other cells312to resize (both their actual size and the size of their associated input control mechanism). Referring toFIG. 7, a user-initiated reconfiguration of one of the collections of cells312illustratively triggers an automatic accommodating reconfiguration of other displayed framework components. This system managed reconfiguration can be for the purpose of maximizing the efficiency with which display space is used, to automatically maintain a logical display of information, or any other controlling principle as desired by an involved system developer and/or administrator.

Up to this point, discussion has for the most part been focused upon broad characteristics of the VisualY model framework164and user-driven navigation through various components of it. What has not been discussed is any particular ontological scheme for organizing the components of the framework. In that regard, it is certainly true that the scope of the present invention is not limited to any one particular scheme. However, at least for the purpose of being complete in terms of the depth of the present description, examples of particular schemes will now be provided.

In one embodiment, each different cell312(loosely represented inFIG. 5by corresponding model segments502) represents a different segment or category of subject matter. An identifier or identifiers (e.g., but not necessarily a symbol or symbols, a color, a sound, an animation, etc.) may be included within a cell312as an indication of the nature of the subject matter relevant to the cell and/or the nature of the subject matter relevant to other cells312navigationally connected to the cell, such as but not limited to a collection of cells312in the next adjacent hierarchical level of the framework. In accordance with one aspect of the present invention, the subject matter distribution across all cells312included in the model framework164is consistent with an overall objective of organizing and presenting enough subject matter cells312to intuitively present a relatively comprehensive and well organized presentation of a broad range of various forms of existence. This is not to say that all manifestations of a model framework164must have this objective. Other framework manifestations could be narrower or even broader in scope. But at least for the purpose of providing a particularly useful example, the broad range of forms of existence will be assumed. This model framework scheme is particularly well suited for the dynamic, interactive, visual and auditory nature of VisualY representations included in instantiations of the VisualY language. The scheme is especially suitable for such purposes because it provides abundant opportunities for showing connections and relationships between various forms of existence.

A good place to begin a description of the exemplary “forms of existence” model framework is to take a more detailed look at the level of the model framework164shown inFIG. 3. The exemplary framework level shown in this screen shot is illustratively the main screen of a human centric view of the “forms of existence” framework scheme. The circled “Y” area326is illustratively the human focus of the model framework. As will be described below in greater detail, a user has the option (e.g., accessible by way of one of buttons314-322) to change to a view other than the human centric view (or to a different variation of a human centric view) and thereby change the presentation and organization of cells312on any or all levels of the model framework164. However, the view shown inFIG. 4is illustratively the primary human centric view.

The cells312on the right-hand side of line306in area310illustratively pertain to fundamental matter/energy/universe categories of subject matter. There are six vertical rectangles, each containing an indication (in this case a symbol or symbols) of the subject matter of the rectangle and/or of sub-cells312included in an adjacent (e.g., connected by way of a direct navigation path) but different collection of cells312, as was described above. Those skilled in the art will appreciate that any of the vertical rectangles are configured to support, as has been described herein, automatic or user initiated re-sizing. In one embodiment, the system is configured such that a user can communicate a re-sizing input (e.g., but not limited to, a two finger, multi-touch pinching open or shut similar to how photographs are currently known to be re-sized) that causes one or more of the rectangles to be increased or decreased in size. It is within the scope of the present invention for any or all cells312(i.e., any of the selectable segments on either side of line306) to be automatically re-sized or re-sized in response to user input.

In one embodiment, the system is configured such that the number of indications sub-cells312included in a given rectangle or any of the selectable segments on either side of line306may change in correspondence with its re-sizing. For example, if the rectangle or another segment is made large enough, symbols representing cells312from multiple cell collections on the same or different framework levels may pop up. The larger the rectangle, the more detail that pops into the rectangle to indicate what other sub-cells or related cells are accessible by way of navigations through the rectangle. In one embodiment, this re-sizing functionality is incorporated into any or all of the framework areas described herein. For example, the cell collection shown inFIG. 4can illustratively be similarly resized automatically or in response to user input so as to change the level of detail presented in terms of what cell collections are accessible through the depicted framework component. Similarly, any of the squares, rectangles, triangles and other shapes within theFIG. 3depiction can be similarly resized. The system is illustratively configured to automatically reorganize a depicted framework representation so as to optimize the use of space and to make the depiction visually appealing and logical, and to present selectable components as easy to select as possible. In one embodiment, this means shrinking, re-representing (e.g., shown as minimized), or deleting one or more framework areas as another framework is increased in size. Or, framework areas can be enlarged, re-represented or re-rendered as another framework area is decreased in size.

In one embodiment, when an area is made large enough, it will eventually sweep in not just the directly accessible adjacent framework levels and cell clusters but also levels and clusters connected beyond the directly accessible. This illustratively means a depiction within an enlarged framework area that distinguishes between those framework levels and/or cells that are directly accessible and those that are not. In one embodiment, not by limitation, this depiction is a representation of a hierarchy such as a tree diagram.

In one embodiment, changes other than re-sizing are also supported. For example, a framework area or a collection of cells312can illustratively be removed or minimized. Similar to how a user of the modern Microsoft Windows operating system is able to minimize a desktop window in order to open up desktop space, a user of the presently described application (or an automated system function), can illustratively minimize one or more portions of the framework164. Or, similar to how a user of Windows is able to “x out” of an application or window, a user of the presently described application (or an automated system function), can close one or portion portions of the framework164. In one embodiment, a “return to default” function is provided to the user as a mechanism for easily undoing adjustments the user has made and returning all or a portion of the framework164to a default state.

Returning again to the description of the exemplary ontological aspects of the “forms of existence” framework scheme, the six rectangles in area310illustratively represent six broad categories of fundamental matter/energy/universe subject matter. The six categories are organized so as to present, to some extent, a right-to-left spectrum of fundamental matter/energy/universe building blocks of which most forms of existence are comprised. The spectrum is illustratively organized such that items classified further to the right are generally more foundational and small while items classified further to the left are generally more complex. The spectrum therefore becomes more complex as it gets closer to the human area326, which is consistent with the fact that humans are often understood to represent one of the most complex forms of existence. Category segments on the left of line306in area308are generally human-oriented while category segments on the right of line306in area310are generally not human-oriented except to the extent that they are building blocks of human-oriented concepts.

In one embodiment, the six categories of subject matter (each represented by a separate one of the rectangles in area310) included in the spectrum, along with exemplary sub-categories (e.g., the sub-categories reflected in associated navigationally accessible hierarchical constructs) are:

1) Primary building blocks (rectangle furthest to right)sub-cells include: fundamental building block components such as but not limited to light, gravity, sub-atoms, energies, forces, etc.

2) Vertical periodic table (rectangle 2nd from the right)sub-cells include: groupings of elements based upon any factor such as but not necessarily limited to function, reactivity, weight, toxicity, value, etc.

3) Non-generative items formed of periodic table components (rectangle 3rd from right)non-generative in the sense that there is generally no contribution to life formssub-cells include: groupings based on any factor such as but not necessarily limited to compounds, mixtures, etc.illustratively sorted from small items up to large items (as large as planets, cosmos, etc. or even larger)

4) Generative things made out of periodic table components (rectangle 4thfrom right)generative in the sense that there is contribution to life formsillustratively sorted from small items up to large items

5) Microorganisms that utilize or comprise the generative things (rectangle 5thfrom right)sub-cells illustratively organized based on size, reactivity (inert or highly reactive thing), etc.organisms in this category generally have no brainincludes access to cells related to environments, etc.

6) Macro organisms that have brain (rectangle 6thfrom right)sub-cells include illustratively organized based on size, capacity, functionality, etc. (humans highest)includes access to cells related to environments, habitats, etc.illustratively includes access to cells related to human built things (cars, compounds, bridges, etc.)

At this point, it is worth noting that mutual exclusivity is not required. For example, a single sub-cell or symbol can be assigned within multiple categories (e.g., can be assigned within more than one “parent” cell312). If a sub-cell or category seems to fit within multiple of the rectangles in area310then it is acceptable for it to appear within multiple of the rectangles. Thus, it stands to reason that a single navigational path can be made accessible from multiple of the rectangles. Further, these principles of non-exclusivity extend to all components of the model framework164. A component of one of a rectangle in area310could also appear in a cell312within area308or vice versa if that makes sense in terms of the ontological logic of the framework scheme or, in particular, if that makes sense in terms of the ontological nature of things that exist in the universe. In one embodiment, the goal is to remain as consistent as possible with the actual organization of the universe, which is a complex endeavor that requires the noted flexible approach.

Before proceeding, it is worth digressing for a moment on the role that reality plays within a VisualY representation, the VisualY model framework, or other components or instantiations of the VisualY language. Within application104, reality is illustratively an adjustable concept. It was stated in the previous paragraph that, in one embodiment, the goal is to remain consistent with the actual organization of the universe. However, in one embodiment, reality is a parameter that may be adjusted programmatically within the software to some extent. In other words, the system provides a function (e.g, made accessible by way of one of buttons314-322or otherwise) that enables the user to selectively define variables in relation to a set of rules defined within the software, the set of rules effectively becoming a particular reality that is imposed programmatically on the configuration of the VisualY model framework and/or as appropriate to other VisualY language components. The default state of the system is illustratively the current human understanding of the universe and everything in it, and thus the default goal is to conform the VisualY language components to that interpretation of reality. However, in one embodiment, the user can adjust to a different reality which may be even be an unrealistic or non-existent reality. This may be as simple as adjusting a rule of physics (e.g., eliminating gravity from the universe) or as fantastical as setting the reality to a fictitious place such as a literary location like the Kingdom of Narnia. In one embodiment, when the user manipulates or otherwise leaves the default reality, the user interfaces of application104are modified to include a clear indicator of the reality in which the system is set. In one embodiment, a mechanism such as a button is provided in order for the user to quickly return the system to its default reality state. In one embodiment, mechanisms are provided within application104to enable a user to create and save new “realities” for later retrieval. For the purposes of the present invention, to the extent that the term “reality” is used in the present description, the term is intended to mean the applicable reality imposed at any given time within application104.

To the left-hand side of line306in area308, the cells312in area326are organized around what is herein referred to as a “yperson” construct, signified by the three lines extending from a central circle so as to form a “Y” shape. As has been discussed the view of the “forms of existence” framework reflected inFIG. 3is human-focused. This view is especially well suited for human cognition at least because humans inherently tend toward a human-centric perspective. However, the model is designed to effectively support VisualY representations of how, given a particular context of interest as selected by the user, the generally human-centric subject matter cells312in area308directly relate to and are connected to the categories of fundamental matter/energy/universe subject matter cells312. This is illustratively an instantiation of the VisualY language. Thus, the VisualY language enables a “big picture” dynamic representation of a complex system, the representation being intuitive and capable of showing simultaneous interplay of multiple or even numerous variables at work within the system.

The “Y” in area326illustratively divides the “yperson” into multiple cells312representing multiple dimensions of the yperson. The cell312on the bottom right identified with a328designation is illustratively the corporeal dimension. WithinFIG. 3, no symbols or sub-cell designations are shown in area328but it is assumed that through this particular cell312the user can see and access sub-cells representing categories of subject matter consistent with corporeal characteristics of the yperson. For example, the cell identified as328can illustratively be expanded (e.g., using a multi-touch “pinch gesture) or navigated to (e.g., navigating to a next level similar to aFIG. 4next level) in order to see more detail about related sub-cells.

The cell312on the top of the “Y” identified with a330designation is the consciousness (awareness) dimension. WithinFIG. 3, no symbols or sub-cell designations are shown in area330but it is assumed that through this particular cell312the user can see and access sub-cells representing categories of subject matter consistent with consciousness-oriented characteristics of the yperson. For example, the cell identified as330can illustratively be expanded (e.g., using a multi-touch “pinch gesture) or navigated to (e.g., navigating to a next level similar to aFIG. 4next level) in order to see more detail about related sub-cells.

The cell312on the bottom left of the “Y” identified with a332designation is the sub-consciousness-oriented dimension. This dimension illustratively encompasses inherent motivations that drive humans. For example, a human may not know what they are about to say until they say it. It is a sub-conscious mechanism that affects what is said. This is but one example. WithinFIG. 3, no symbols or sub-cell designations are shown in area332but it is assumed that through this particular cell312the user can see and access sub-cells representing categories of subject matter consistent with sub-consciousness-oriented characteristics of the yperson. For example, the cell identified as332can illustratively be expanded (e.g., using a multi-touch “pinch gesture) or navigated to (e.g., navigating to a next level similar to aFIG. 4next level) in order to see more detail about related sub-cells.

The cells312in the small corners362around the “Y” illustratively represent the yperson's applicable environments. WithinFIG. 3, no symbols or sub-cell designations are shown in corners362but it is assumed that through these particular cells312the user can see and access sub-cells representing categories of subject matter consistent with environmental considerations relative to the yperson. For example, the corner362cells can illustratively be expanded (e.g., using a multi-touch “pinch gesture) or navigated to (e.g., navigating to a next level similar to aFIG. 4next level) in order to see more detail about related sub-cells. In one embodiment, each of the corners362(and associated sub-cells312, etc) represents a different “level” of environmental considerations, illustratively on a spectrum from micro to macro (e.g., from the micro environment to macro environment).

Above the yperson area326are a plurality of horizontal rectangles in a non-tangible human construct area340. Each rectangle in area340is illustratively a different cell312, although only a representative few have been labeled in theFIG. 3. The cells312in area340illustratively represent, within the model framework, non-tangible “things” created by humans. For example, it is through a cell312that the user might access a cell312representing the culture of a particular city (e.g., but not necessarily depending upon user-selected application preferences, the culture of the city where the currently selected ypserson(s) live). Other cells also lead to cells312that represent how people think (e.g., but not necessarily depending upon user-selected application preferences where the currently selected yperson(s) lives). Other examples include what is believed, how the yperson(s) have organized (institutions, etc.), normals, rules, laws, governments, etc. Of course, these are only examples and the scope of the present invention is certainly not so limited.

Notably, the rectangles in area340, which generally represent non-tangible things, are horizontally oriented. In contrast, the rectangles in area310, which generally represent tangible things, are vertically oriented. In one embodiment, this theme is carried throughout the application interfaces. In one embodiment, a rectangle oriented at an angle represents something where the tangible/non-tangible line is ambiguous or not clear.

To the left of the yperson area326are a plurality of circles in a people area364. Each circle in area364is illustratively (though not necessarily) a different cell312, although only a representative few have been labeled in theFIG. 3. The cells312in area364illustratively represent, within the model framework, people having a relationship to the yperson, or at least having interaction on some level with the yperson. Each circle illustratively represents one person or a group of persons. The representation may be very specific (e.g., a circle representing single specific person such as President Bill Clinton) or may be collective or otherwise generic (e.g., a circle representing farmers from Kansas). Some cell312circles included in or accessed through area364(e.g., sub-cells312made accessible by way of navigation through area364cells) will represent individual humans (e.g., Paul Johnson from Toledo, or a postman, etc.) and others will represent groupings of humans (Boy Scout troop345from Toledo, or communists, etc.).

Those skilled in the art will appreciate that just as the system is configured to enable the user to navigate through the rectangles in area310(or through any other cell312clusters in the model framework), so too can the user navigate through the circles in area364, for example to other circles or even to portions of the model framework outside of area364if logically appropriate. It is therefore to be understood that all of embodiments described herein at least in terms of the use of symbols, navigation methods and the like described in the context of other cells and cell clusters in the model framework are just as applicable to the circles in area364. Still further, just as vertical rectangles and horizontal rectangles used within the model framework signal relevance to a particular area within the ontology of the model framework, so too does the use of circles illustratively signal a connection to a people-oriented relationship to the yperson.

It is within the scope of the present invention for a circle or circles within the model framework to represent living people or dead people. Circles may represent people that exist or people that do not exist or never have existed (e.g., fictional people). They may represent people from the past, present or future. Further, the term “people” should be liberally construed. Circles may also represent cartoon characters (e.g., Scooby Doo), animals (e.g., a favorite pet), or any other person, place or thing that has the capacity to function in a manner similar to a person in terms of their interaction with, or relationship to, the yperson. By not strictly requiring a circle to represent a living human, the system is flexible enough to accurately reflect the reality (the reader should be reminded that, as state above, reality is a programmatically adjustable concept within application104) of even an unusual set of circumstances. Further, as has been described, the system is also flexible enough to allow an entity represented by a circle to also be represented elsewhere in the model framework. The goal is, in one embodiment, to configure the ontology of the model framework so as to accurately reflect reality (the reader should be reminded that, as state above, reality is a programmatically adjustable concept within application104). Thus it is undesirable to impose rules that might simplify the model framework at the expense of departing from reality.

In one embodiment, the configuration and organization of the cells312in area364may be but illustratively is not random or arbitrary. Instead, the configuration and organization of the circles in terms of their placement relative to each other, their placement relative to the top or sides of area364, their placement relative to yperson area326, their motion, their color, their associated sounds, their size, or any other adjustable parameter is indicative of a characteristic of a relationship or level of interactions with the yperson. For example, in one embodiment, the directness of an interaction is factored into the scheme utilized to organize the cell312circles. Humans with the most direct interaction are illustratively touching or at least closest to yperson area326.

In another embodiment, the depiction scheme of circles within the model framework includes a mechanism for indicating whether an exchange with the yperson is direct or indirect. For example, an exchange mediated through another entity is illustratively accounted for in the depiction scheme. In one embodiment, assuming such a scenario is consistent with reality (the reader should be reminded that, as state above, reality is a programmatically adjustable concept within application104), a mediated exchange may even be made shown in relation to a circle representing someone no longer living. For example, if a now dead person that never met the yperson made music years ago that is listened to by the yperson that can be considered a mediated exchange and is illustratively shown as such within the model framework. In one embodiment, mediated exchanges are indicated by utilization of a slightly different shape (e.g., an oval instead of a circle), color, sound, placement, etc. Further, less direct exchanges (such as but not limited to mediated exchanges) are illustratively assigned relatively smaller cells312and/or are positioned further away from the yperson within the model framework. Of course, these are only examples and the scope of the present invention is certainly not so limited.

Below the Y area326is an area366dedicated to depicting an applicable spatial action or otherwise a context for an active VisualY representation. In other words, area366provides a way to show an accurate context over space and/or time for the currently active VisualY representation. For example, a video of a person jumping out of a plane may be shown in area366while, as part of an applicable instantiation of the VisualY language, many cells312throughout the VisualY framework light up, change color, move, generate sound, become associated with an animation or symbol, and otherwise show, on the potentially many relevant levels of the framework, what is happening to the yperson while they are jumping. Of course, this is but one of an unlimited number of examples of how area366is utilized to indicate a context for activity (e.g., sounds, coloration, changes in sound, changes in color, animations, etc) occurring in relation to cells312in the framework164.

In one embodiment, the context provided in area366is active so as to depict a dynamic context for changes occurring relative to the framework as part of a VisualY representation. Thus, the provided context can be a video, or an animation, or a series of photos, or an arrow or other pointer moving across any of a variety of different types of timelines (e.g., a human development timeline, a date-oriented timeline, or a timeline organized around any other concept). Alternatively, the context provided in area366can be static such as a picture or a map. The context provided in area366may be made in terms of the places or events in the past, present or future. It can be a depiction of events that did already occur, events that have never occurred, events that will but have not yet occurred, events that will never occur, or otherwise. The context may be presented in terms of real places or places that do not exist in the “real” world, or even fictitious places. Or, it can be an event such as World War II or Super Bowl VII. It is within the scope of the present invention for the context provided in area366to be absolutely any mechanism for depicting a space, time, subject matter or any other context for what is being represented in relative to the framework164as part of an active instantiation of the VisualY language (i.e., an active VisualY representation).

In accordance with another aspect of the present invention, just like other cell312areas within the framework are configured to support navigation to other levels of detail by navigating to other cell312areas, so too may area366be configured to support navigation to other levels of detail. In one embodiment, the system is configured to enable a user to choose from a variety of different perspectives of the same context for a VisualY representation. For example, the user might navigate from a pulled back video view of a person sitting on a bench eating corn to a close-up video view of the person's jaw chewing corn. In one embodiment, the system illustratively includes mechanisms for slowing down (e.g., slow motion), speeding up, re-starting, rewinding, pausing, or otherwise manipulating a video or animated area366context, thereby illustratively causing a corresponding (e.g., slowing down, speeding, pausing, rewinding, etc.) change to the current dynamic VisualY representation. These are but examples of how the area366area context can be selectively adjusted without actually changing the active VisualY representation.

However, it is also within the scope of the present invention for the system to be configured to utilize navigation of area366(or similar functionality accessed elsewhere within the system such as through functions associated with one of buttons314-322) as a mechanism that enables the user to switch from one context for an active VisualY representation to a different context for the VisualY representation, thereby effectively transitioning from one VisualY representation to another. For example, the context area366might be changed from one person eating corn in a “right now” sense to a specific infant child eating corn for every meal for a month. This would not only change the context for the currently active VisualY representation but would also change the VisualY representation itself (e.g., it would change the yperson area326from the one person yperson focus to the infant child focus, along with the many other likely changes to the VisualY representation in relation to model164that that would produce).

When the context area366is navigable and/or alterable, the system illustratively provides an indication such as a symbol, color or sound to indicate navigability. In one embodiment, area366is navigable and otherwise can be manipulated in at least all of the same ways described herein in relation to cells312included within the framework164. For example, different context options might be accessible through area366in the form of cells312made accessible through area366. In one embodiment, one or more cells (even potentially multiple cells simultaneously) accessed through area366can be selected so as to set the active context for the currently active VisualY representation.

In one embodiment, any or all of the cells312in framework164can be made selectable such that, when selected by the user, corresponding changes are instantly made to the currently active VisualY representation. Thus, the user is illustratively able to navigate throughout the framework selecting and/or deselecting cells312as desired and thereby configuring the currently active VisualY representation as desired. In one embodiment, the user can start an active space/time video context in area366at the beginning of a video that is the context for a first VisualY representation that is made while the video is playing. Then, the user can pause the video while navigating through model164and making changes by selecting and de-selecting cells312. Then, the user can manipulate a control provided within the system in order to re-start the video. An updated VisualY representation will then be shown in relation to the re-started video, the representation being updated due to the new set of variables applied to the representation by way of the new combination of selected and de-selected cells312. In other words, the behavior of the VisualY representation (e.g., colors, sounds, animations, motions, etc. shown in relation to the framework164) is programmatically dependent upon system variables as selectively set by the user of the application. In one embodiment, the behavior of the VisualY representation is also programmatically dependent upon which view of all or a portion of the framework164the user has selected.

4. The VisualY Engine

As should be becoming apparent, application104is configured to provide a richly dynamic and programmatically responsive user experience in the form of instantiations of the VisualY language. A given instantiation is likely to include a VisualY representation of information, which illustratively includes sound, color, multimedia, motion (or any other similar means for conveying information) provided in relation to any or all of the components of the VisualY framework. As the user navigates through the framework, as the user selects and/or deselects components of the frameworks, and as the user changes views of all or portions of the framework, the relevant VisualY representations (i.e., color, sound, multimedia, motion, etc.) are configured to programmatically respond and be presented to the user in a way that communicates knowledge and information in the context of the currently presented VisualY framework components. It should also be noted that, in one embodiment, other programmatic variables may come into play, such as other user-selected parameters (e.g., in one embodiment, the user selects a purpose for their current exploration of the VisualY instantiation) that will impact how the framework and/or representation responds programmatically.

In accordance with one aspect of the present invention, the application104component that drives the dynamic responsiveness of instantiations of the VisualY language is a VisualY engine170, which is shown inFIG. 2. It is a misnomer of sorts to refer to engine170as a single component because, in implementation, it is likely comprised of a variety of different hardware and software sub-components.

Engine170includes, in one embodiment, front end software components that drive the front end of application104. For example, the front end components determine the user experience in terms of the sounds, images, multi-media, content, functions, controls and tools presented to a given user automatically, semi-automatically or in response to user input or otherwise. In one embodiment, the front end components include primarily JavaScript coded components that communicate with backend system components responsible for actually retrieving information from the database(s) as necessary to appropriately fulfill a frontend requirement.

In one embodiment, the front end software components of engine170include WebGL programmed components that extend the capability of the JavaScript programming language so as to enable engine170to generate interactive application104graphics (e.g., interactive 3D graphics) within the browser of users of the application. This WebGL code illustratively executes on the computer display cards of the users. Those skilled in the art will appreciate that WebGL is a context of the canvas HTML element and, accordingly, components of the front end software components of engine170are coded in accordance with canvas programming constructs.

In one embodiment, the front end software components include audio API components that extend the capability of the JavaScript programming language so as to enable engine170to dynamically generate interactive application104audio in logical relation to visual information presented through the user's browser. The audio API components enable manipulations (changes in pitch, tempo, etc.) of audio signals on the sound cards of the users. Accordingly, in one embodiment, the audio is synthesized and processed directly in JavaScript without use of a plug-in. In one embodiment, audio is managed through the AudioData API (Mozila Firefix) and/or the WebAudio API (Chrome and Sarari) thereby enabling audio information to be manipulated more directly or at least relatively more directly upon the user's hardware responsible for audio data management.

In one embodiment, certain of the front end software components, including the components for managing audio and video features presented to an application user, are coded so as to be conditional in the sense of being rules based but are not so conditional as to require coded anticipation of every possible outcome. The described JavaScript API's are illustratively applied within the engine170framework so as to enable a user of application104to manipulate variables (e.g., manipulate manifestations of the VisualY language by selecting de-selecting, or navigating through sets of cells312) that affect in essentially real time what is presented to them in terms of their audio and visual application experience. The JavaScript API's enable this to be accomplished without working through a browser plug-in such as a Flash plug-in. By enabling the audio and visual experience to be efficiently managed and manipulated by the user's own audio and video hardware, the JavaScript API's enable the front end components to provide what is essentially a generative user experience. In one embodiment, the JavaScript API's are utilized within the front end software components so as to provide a visual and/or an audio user experience powered by some degree of programmatic artificial intelligence (or at least an allusion from the user's perspective of artificial intelligence) instead of a strictly rules based logical approach. Thus, not every possible outcome (e.g., not every possible combination of activated or deactivated cells312) need be programmatically accounted for per se in order for there to be a logical (at least from the user's perspective) manipulation of the user experience.

It should be said that these generative components of driver170may still technically be conditional in terms of their logic but they also sometimes will programmatically accommodate complexity or chaos to some degree. In one embodiment, the algorithms utilized to drive the experience of a user of application104facilitate production of user experiential outcomes that were not strictly programmatically anticipated when the code was originally developed. In another embodiment, the system is programmed to have an ability to “learn” that is implemented through a variety of artificial intelligence algorithms and techniques. Then, an outcome that is learned may illustratively be re-created differently under a same or similar set of conditions subsequently established by the user through manipulations made during interaction with the application104.

To summarize, the experience of the user of application104as facilitated by engine170is interesting in terms of how it is delivered through the user's web browser on a user-driven basis (i.e., responsive to user adjustment of application variables) so as to be dynamically generative and responsive thanks at least in part to direct generation and/or manipulation of audio and/or visual data upon (or at least in a close direct relationship with) the user's audio and video hardware. This is illustratively accomplished without reliance upon functionality delivered by way of a web browser plug-in (e.g., a Flash plug-in) or some other similarly delivered additional functionality. While the scope of the present invention does not exclude a plug-in driven implementation such as a Flash-based implementation, such a solution is assumed to be less desirable, for example, because the graphics throughput is considerably less and there will generally be limited or no graphics shading options. The options for presenting representations of information-rich, dynamic complex systems, and for enabling multiple people in remote locations to collaborate in real time in the continuous development of such representations, are far greater and more flexible utilizing the non-plug-in implementations described herein than the plug-in driven solutions that, at the time of this writing, currently are the dominate choice by far for navigating the more dynamic and “alive” sites on the world wide web and on other similar informational networks.

It is worth elaborating some on the collaboration aspect of embodiments of the present invention. The experience of the user of application104as facilitated by engine170illustratively revolves around a community that collaborates essentially in real time. Engine170is configured to enable the users of application104to collectively and remotely create, organize, collect, modify, transform, present, navigate, browse, search, experience, display, store, retrieve, transfer, and otherwise interact with instantiations of the VisualY language. In one embodiment, the VisualY representations162(e.g., presented to a user in the context of all or a portion of the VisualY model framework164) are made available on the World Wide Web such that the described and other functions of engine170are made publicly available to some extent.

Accordingly, embodiments of the present invention pertain to applications of relatively new browser technology that enable relatively large graphics (and audio) throughput. The system enables a connection of easy to use and flexible programming code to an especially efficient pipeline to a user's hardware where it is quickly executed. This direct pipeline scheme enables much more throughput than a traditional approach where code is first moved around and processed through higher levels within the computing architecture. For example, the direct pipeline approach enables the skipping of more traditional steps such as setting up binary and library resource requirements, compiling to binary, fixing cross operating system parameters, etc. The described browser-driven but not plug-in driven programmatic approaches (such as WebGL) are essentially pathways to getting code directly onto hardware, thereby making it much faster to execute. This technology is directly responsible, in accordance with one aspect of the present invention, for empowering the user community to directly or indirectly participate in a collaborative effort to manipulate the code (e.g., manipulate instantiations of the VisualY language) and save progress as such manipulations are made. The users thereby gain from each other as value (i.e., in the form of information and knowledge communicated in accordance with the constructs of the VisualY language) is added and modified on a collaborative basis. And this is still done utilizing a browser driven remote application approach rather than another approach, for example, requiring each user to obtain a stand alone application.

In one embodiment, the VisualY language includes a code based framework with styling principles, macro functionality for task accomplishment, web failbacks for cross browser support, etc. The engine170illustratively utilizes these and other components of the VisualY library and framework. The engine is an instantiation of code that illustratively comes from the VisualY library and framework. Thus, it can be difficult to draw exactly the line between the language and the engine. However, exactly where this line lies is not necessarily critical to the present invention.

FIG. 8is a block diagram showing, in accordance with illustratively embodiments of the present invention, components of engine170. Engine170is illustratively an instantiation of VisualY core components802. In essence, engine170is the driver and has its hands in (and relies upon) the functionality of the core components802. In one embodiment, core components802provide the visual language manifested as JavaScript (though the scope of the present invention is not so limited). These components are illustratively the API's of the application104system and therefore define the basic application components.

Included under the core components802are DOM (document object model) utilities804. This is illustratively where information is provided for manipulating the page structure within application104. These components are illustratively coded as CSS3 and/or HTML5 constructs, though the scope of the present invention is not so limited. Also included are data utilities806, where functionality is provided for manipulating data that is to be presented within application104.

Also included under the core components802are charting components808. The charting components illustratively include, but not by limitation, weighted stacks, bar charts, model handling functions, etc. This may or may not be where rendering is handled but is illustratively an area where instructions are recorded to define how things are drawn. A separate sketching package810provides functions that make it possible to draw on a very efficient basis, as has been described in relation to the incorporation of WebGL and Canvas mechanisms into the application architecture. A set of styling components812define available colors, line thicknesses, etc. Those skilled in the art will appreciate that the description of components of engine170described herein is not exhaustive and that other components are likely to be included at least to supply basic functions that are known in the art to be part of the kind of network driven application architecture described herein. For this reason, a catch-all box814is included in theFIG. 8depiction of engine170.

5. First Principles Engines

In relation to the VisualY model164it was explained that cells312within the model framework need not be exclusive in order to enable an accurate, scientifically founded presentation of information and knowledge. This is almost certainly true of the cells312in area326where interplay between the different dimensions of the yperson are likely to be common. For example, the line between conscious processes and subconscious process is not necessarily going to be clear. In order to maintain the integrity of the ontological classification scheme, it will sometimes make sense for there to be some level of redundancy in terms of the components of one cell312relative to another.

As much as possible, this interplay and overlap within the model framework164is preferably managed not on an arbitrary or opinionated but instead is managed so as to remain as consistent as possible with the interplay and overlap of corresponding processes that exist in reality. In other words, as the model framework grows and evolves, it is illustratively the goal to stay as consistent as possible with the way in which humans, nature and other manifestations of existence are actually programmed. For example, if a human wants to control something, then an awareness of the thing to be controlled must first be created. Humans generally have some degree of control in terms of their view of what is really happening within their surroundings. So, the line between the conscious and sub-conscious is not necessarily always clear. By not requiring a given cell312construct to be assigned exclusively to a single location within the model framework164, there is allowance for enough flexibility to address mysteries or ambiguities or complexities in terms of the underlying programming and processes that drive reality (the reader should be reminded that, as state above, reality is a programmatically adjustable concept within application104).

The goal of reflecting and representing real underlying fundamental programming and processes is not exclusive to the human context of the model framework. As is likely becoming apparent, the VisualY model framework is intended to encompass all levels of existence, from the smallest building blocks to the largest most complex constructs. In reality (in particular the default reality of the universe discussed above in the context of programmatically adjusting reality within application104), every manifestation of existence is subject to some degree of inherent fundamental programming that dictates, to some extent, interactions among and between the constructs represented within the model framework. By not requiring a given cell312construct to be assigned exclusively to a single location within the model framework, there is allowance for enough flexibility to address mysteries or ambiguities or complexities in terms of the underlying programming and processes that drive reality (though the reader should be reminded that, as state above, reality is a programmatically adjustable concept within application104).

In accordance with one aspect of the present invention, first principles engines816, which are shown inFIG. 8as part of the larger engine170, function within the application104framework as the mechanism for enforcing restrictions and requirements relative to the VisualY model framework and related VisualY representations (i.e., instantiations of the VisualY language) based on scientific, nature, and/or other fundamental principles. For example, the principles upon which the restrictions and requirements are based are illustratively first principles. A first principle is, in a philosophical sense, a basic foundational proposition or assumption that cannot be deduced from any other proposition or assumption. To the extent that new first principles are discovered or existing first principles are abandoned or changed, corresponding changes can be made (for example, by an administrator of application104) to the engines816in order to keep, as much as possible, a given VisualY instantiation consistent with a fundamentally accurate portrayal of all levels of existence.

In one embodiment, engine816will, on the basis of maintaining alignment with a first principles or other principle accounted for in engine816, require or refuse a cell312, a link between cells312, a color designation, an animation, etc. Thus, engines816are utilized within application104to enforce alignment with a set of bases principles by policing, to some extent, interactions among and between the constructs represented within the VisualY model framework.

In reality, everything (i.e., all things at all levels of existence) is in a constant state of change. Consistent with this premise, instantiations of the VisualY language provides a visual (and sometimes audial) representation of all constructs of existence and the VisualY programming and language provides the tools for showing the constant change and interactions among the constructs of existence. And a fundamental assumption underlying this approach is that everything in existence is in a constant state of change and everything in existence is, to some extent, subject to some degree of fundamental programming that drives the change and therefore directly influences interactions at all levels of existence.

Thus, when the mechanisms of VisualY are applied within a given context to model framework164, it becomes possible to explicitly depict human programming and interactions among the various dimensions of the yperson (i.e., the “person” reflected in area326). In one embodiment, the application includes a function (e.g., a function accessed by way of one of buttons314-322, or by navigating through the constructs312of the model framework and selecting a cell312or setting parameters in relation to a cell312) that enables a user to select a time independent or time dependent context to be imposed upon the VisualY representations of interactions and programming displayed in relation to the model framework. For example, the user can set the display to show what is happening now, what happened during a point in history (a specific date or a range of time, etc.), or what will happen in the future. However, selecting a time context is not limited to units of time as measured in relation to a clock or calendar. The user can illustratively choose any measure of time such as an historical event (e.g., during World War II), a human event (e.g., during birth, during the yperson's second decade of life, after the yperson eats, after the yperson plays football for 10 years, etc.), or even a theoretical event that is not consistent with reality (e.g., after the yperson eats corn, or after the yperson gets a concussion, etc.).

As is apparent in some of these examples, the application is illustratively configured to enable a user to select and/or set any of what is likely to be an abundant number of parameters that have the potential to impact the VisualY representations of interactions and programming displayed in relation to the model framework. While these parameters may be set by way of functions accessed by way of one of buttons314-322, it is more likely that such parameters will be set by navigating through the constructs312of the model framework and simply selecting a cell312or setting parameters in relation to a cell312. The functions that dictate how different parameters will affect the model framework and/or the VisualY representations (e.g., how interactions between model framework constructs312are shown using the mechanisms of the VisualY programming and language) is illustratively built into the computer code underlying the framework and/or the mechanisms of the VisualY programming and language.

Accordingly, the model framework164and the mechanisms of the VisualY programming and language combine to make it possible, for example, to depict what gets into the memory of the yperson (i.e., knowledge). And this can be seen at different levels and with connections to other dimensions of the yperson and with connections to other constructs outside of the human realm (e.g., with connections to constructs included in the subject matter categories of the cells312on side310of line306). Further, it is also possible to depict, for example, specific types of trauma within appropriate cell312dimensions of the yperson and with explicit and appropriate connections to other cells312on any level of the model framework164.

In one embodiment, the system enables the user to set a particular yperson context for area326. This may be accomplished through interaction with a function accessed by way of buttons314-322. Also or alternatively, setting the yperson context may be accomplished by navigating through the constructs312of the model framework and selecting a cell312or setting parameters in relation to a cell312. The yperson context can illustratively be set to represent a specific person or a generic human being or a group of human beings (e.g., a company of individuals, an organization of individuals, etc.). When the ypserson context is changed, the rest of the model framework164and its related VisualY programming illustratively updates automatically in order to modify the representation of complexity so as to be consistent with the selected yperson focus.

In one embodiment, which is visually depicted inFIG. 3by the overlapping circle334, the yperson focus of the representation can be a group of persons each having their own yperson representation. The system is illustratively configured to enable the user to selectively add or remove ypersons to the “stack” of “Y” people in area326. As users are added or removed, the rest of the model framework164and its related VisualY programming will illustratively update automatically in order to modify the representation of complexity so as to be consistent with the selected yperson focus. Thus, depending upon the selected settings in area326, the system is designed to account for and represent information in relation to collective (i.e., across a group) human dimensions (e.g., collective consciousness, sub-consciousness, etc.). Similar to the described functionality described above in relation to a single yperson context, the mechanisms of the VisualY programming then trigger representations of connections between the human-oriented cells312on side308of line306and the other more foundational cells312on side310of line306.

In one embodiment, the yperson focus in area326indicates a group of different individuals the application can be set to only conform the representation to one of the individuals at a time. For example, the user can illustratively see layers if individuals represented in the rings within area326(e.g., the top-most yperson representation is illustratively rendered semi-transparent so other y-persons underneath are apparent, or the edges of other y-persons might appear so as to represent a stack of ypersons). The system is illustratively configured to enable the user to switch between different ypersons or groups of ypersons as desired, which will illustratively initiate appropriate and corresponding changes within the VisualY programming and language representations in relation to the model framework.

In one embodiment, the yperson focus can be set to a class of people or a very large number of people. For example, can illustratively set to all republicans in the United States, or all people in China, etc. Also or alternatively, it can illustratively be set to a hypothetical but not necessarily specific person, such as an “open-minded person” or “a Christian.” In one embodiment, the yperson can be set to be the “average” person, determined by all people that the user has included in a “stack” of ypersons within are326. In this case, the selected yperson would be a conglomeration of all persons included in the stack.

In one embodiment, the selected yperson can even be selected to be a non-human but living creature. For example, the yperson can be selected to be an ant, or a breed of dog, or a specific dog. Any living thing can, in one embodiment, be chosen as the focus of the representation within areas326. However, the focus of the representation within areas326can be something other than a living thing. It can be a deity, a fictional character, an organization, a fictional organization, or anything else.

As has been hinted at, it is to be understood that some or all of cells312support some degree of data input. For example, when the yperson context is set to a specific individual, it might be implemented for a purpose such as that person's medical chart. Both the doctor and the patient (i.e., the individual) can illustratively interact with the model framework164utilizing mechanisms of the VisualY programming and language so as to add information to the framework and thereby to the chart. For example, the doctor might navigate through the model framework and add information in an appropriate cell312location regarding the result of lab tests for the individual. The individual might navigate through and add, in an appropriate cell312location, information as to how they are feeling and when they were feeling it, or what they ate and when they ate it, etc. Then, in accordance with one aspect of the present invention, the model framework is programmed to some extent to respond to the entered information by automatically implementing VisualY mechanisms that depict appropriate and logical connections to other cell312constructs.

Accordingly, the model framework164and the mechanisms of the VisualY programming language combine to make it possible, for example, to represent complexities of a human state of being (even during any desired user-selected period of time, as described above). Further, the direct human aspects of the state of being (i.e., shown in relation to cells312on side308of line306) are represented in direct relation to more fundamental aspects of the state of being (i.e., shown in relation to cells312on side310of line306). Thus, in one embodiment, it becomes possible to show what foods were eaten in combination with even the most fundamental components of those foods, and what impacts eating the foods have on the yperson.

It is worth emphasizing that the model framework components and the related VisualY language and programming representations will, by design, vary based on commands and preferences received from the user. The user will illustratively only see the model framework level(s) and the level of VisualY detail that they want to see, which very well may include all levels. The application is illustratively configured to enable the user to hide, fold up, or re-size cells312as desired to cause the application to display a desired level and scope of detail.

VisualY representations when applied to the model framework164enable users to see how things on all levels of nature interrelate to one another. Human constructs and ideas in one part of the model framework are actively shown (i.e., are shown in terms of their inherently dynamic and active nature) in relation to interrelated principles of the scientific foundations of the world in another part of the model framework. This reflects the fact that everything is programmed in nature, from the smallest things to the largest. Human beings have the capacity to understand the related mechanisms that drives us and that drive everything. In order to tap into this understanding, one must move away from seeing things superficially and from a very limited point of view. Instantiations of the VisualY language enables a new and better perspective of how things really are in nature.

Instantiations of the VisualY language reflect the fact that forms of existence, including but not limited to the forms of existence represented within framework164, have at their root energy that drives atomic and subatomic existence, that drives light and gravity, that drives building block elements and forces, that drives what humans refer to as life. Everything is in play—doing something at all times—activated. Everything is dynamic. Everything is complex in that everything is, to some extent, a product of complex interaction.

Thus, instantiations of the VisualY language reflect the fact that there is a dynamic dimension to everything. Nothing is static. While instruments of science are sensitive to the constant state of change, humans on their own have a difficult time sensing it due in large part to limitations of the human senses. It should therefore probably come as no surprise that human language has not developed so as to be particularly effective when it comes to describing the dynamic and interactive dimensions of human surroundings. And because humans relay on language and senses almost exclusively for our understanding of the existence around us, and within us, we have an extremely limited day-to-day, theoretical, practical comprehension of how everything in existence works. Current systems of language consist mainly of words and do not easily enable one to fully and accurately imagine or describe different forms of existence. Current systems are inadequate at least in that they are inherently linear and for the most part two-dimensional.

FIG. 9is a screen shot of a multi-variable system for producing user interface components on a generative basis. On the left hand side of the screen shot are a plurality of user adjustable variables. Each variable is capable of being set to any of a range of different states. Changing the state of a variable will illustratively change the motion, color, sound, and otherwise the configuration of patterns on the right side of the screen shot.

In one embodiment, patterns such as those on the right hand side of theFIG. 9screen shot are examples of what parts of a VisualY representation will look like. Thus, one or more of the patterns on the right hand side are depicted in conjunction with (e.g., over one or more cells) the VisualY model framework as part of an instantiation of the VisualY language. Those skilled in the art will appreciate that, when the patterns are incorporated into an instantiation, the changing of the variables may be user-initiated but some or all of them are also automatically adjusted programmatically. For example, the generative nature of the programming of engine170will illustratively drive changes in the variables and therefore changes in the patterns and therefore changes in the instantiation.

FIG. 10is an example screen shot demonstrating an alternate view of a portion of the VisualY model framework. In one embodiment, the gridded area on the right hand side of the alternate view is a substantially or entirely complete depiction of the VisualY model framework, even if the components depicted on the left hand side are only a portion of the model framework. Instantiations of the VisualY model framework are illustratively presented as VisualY representations presented in conjunctions with the model framework as shown inFIG. 10. In one embodiment, the user is able to move different combinations of sub-portions of the model framework in the left hand panel while at the same time always seeing a presentation of the more or fully complete model framework in the right hand panel. Instantiations of the VisualY language are then active in that VisualY representations are responsively and generatively presented in either or both of the panels in conjunction with the different depictions of the VisualY model framework.

FIG. 11is another screen shot of a multi-variable system for producing user interface components on a generative basis. In the small box in the top right hand corner of the screen shot are a plurality of user adjustable variables. Each variable is capable of being set to any of a range of different states. Changing the state of a variable will illustratively change the motion, color, sound, and otherwise the configuration of patterns shown in the rest of the screen shot.

In one embodiment, patterns such as those shown in the bulk ofFIG. 11are examples of what parts of a VisualY representation will look like. Thus, one or more of the patterns are illustratively depicted in conjunction with (e.g., over one or more cells) the VisualY model framework as part of an instantiation of the VisualY language. Those skilled in the art will appreciate that, when the patterns are incorporated into an instantiation, the changing of the variables may be user-initiated but some or all of them are also automatically adjusted programmatically. For example, the generative nature of the programming of engine170will illustratively drive changes in the variables and therefore changes in the patterns and therefore changes in the instantiation.FIG. 12is a screen shot of a user interface for selecting and thereby initiating display of an instantiation of the VisualY language.

Accordingly, embodiments of the present invention pertain not just to data visualization but to knowledge visualization. In one embodiment, all instantiations of the VisualY language are supported, meaning assumptions in the VisualY model framework and/or the VisualY representations, by proofs that are may be accessed, for example but not by limitation, by navigating (e.g., navigating to a lowest level) through the VisualY model framework as described herein. The proofs illustratively provide source data to back up the assumptions in terms of how constructs are related and presented as being connected.

In one embodiment, relationships such as but not limited to data relationships are embedded in the coding of the VisualY language. Thus, the system is generative at least in that the applications engines use data to determine, for example based on first principles as described herein, what the programmatic response of the VisualY instantiation will be. In essence, the behavior of a manifestation of the VisualY language over time is illustratively generative in that it is data dependent—and those dependencies are designed with the intent of being as consistent as possible with first principles of science and nature. The dependencies are illustratively adjusted as necessary to be as accurate as possible. Thus, manifestations of the VisualY language are much more than simple data visualization. The manifestations are such that data is truly relational in that it is built into the programmatic assumptions of the coding.

Further, embodiments described herein go beyond known methods and systems in the field of semiotics. Embodiments described herein involve cognitive semiotics. Embodiments involve a semiotic coding system syntactics and cybernetics. The systems in which knowledge is presented in accordance with instantiations of the VisualY language are organic in that they are dynamically generative and not necessarily predetermined or static. Embodiments are robust and scalable.

In one embodiment, the VisualY language is organized around an objective driven concept. What the user sees in an instantiation of the VisualY language is what they want. The user essentially makes selections and navigations so as to adjust the instantiation and effectively filter the user interface down to what they want.