Identifying boundaries of substrings to be extracted from log files

Described herein are various technologies pertaining to identifying boundaries of a substring in a log file, wherein the substring is to be extracted from the log file and used to construct a table. An indication is received that a user has selected a beginning boundary of the substring, and the ending boundary of the substring is automatically identified.

BACKGROUND

As computing devices have become ubiquitous, the volume of data produced by such computing devices has continuously increased. Organizations often wish to obtain insights about their processes, products, etc., based upon data generated by numerous data sources, wherein the data from different data sources may have different formats. To allow for these insights to be extracted from data, the data must first be “cleaned” such that a client application (such as an application that is configured to generate visualizations of the data) can consume the data and produce abstractions over the data.

In an example, server computing devices of an enterprise can be configured to output log files. These log files have a “flat” structure, in that a log file does not contain a (hierarchical) presentation of the data included in the log file (unlike a JSON document or an XML document). Further, log files tend to comprise unstructured or semi-structured data, rendering it difficult to analyze such data in its native form. For instance, an application executing on a server computing device can generate a log file that indicates times that particular actions were undertaken by the server computing device when executing the application. Data lines in a log file, however, may include semi-structured data, such that executing a query over the log file is problematic. Hence, it is often desirable to extract certain data from a log file and place the data in tabular form, such that a client application can then further process the data using standard tabular analysis tools.

Conventionally, it is cumbersome to extract data from log files and place it in tabular form. One exemplary approach is for a user (e.g., a data cleaner) to manually extract desired data from a log file and placing the extracted data in appropriate cells of a table. Log files, however, may include thousands to millions of lines of information and, therefore, this manual approach is often not possible. Another exemplary approach is for a programmer to write a script that extracts data from the log file and populates cells of a table based upon the data extracted from the log file. This approach, however, requires programming expertise. Further, different applications generate log files with different data structures; therefore, writing the program often is a one-off project, which is an inefficient use of programmer time.

Relatively recently, programming by example (PBE) technologies have been developed, where programs are synthesized based upon examples provided by end users. The structure of most log files, however, is not well-suited for PBE technologies. More specifically, log files tend to have various different types of lines therein, including but not limited to header lines, comment lines, and data lines. Thus, conventionally, an end user may be required to explicitly identify lines (such as comment lines and header lines) that do not include data that is of interest to the end user as negative examples. Further, for conventional PBE technologies to be employed to synthesize a program that is configured to extract data from a log file and place it appropriately in a table, the end user must explicitly identify boundaries of records in the log file. This may be burdensome for the end user, as the task of identifying record boundaries may not match the mental model of the user, who may simply care to extract certain fields.

SUMMARY

Described herein are various technologies pertaining to constructing a table based upon a log file output by a computing device. The technologies are particularly well-suited for use in connection with PBE technologies, which are configured to synthesize a program based upon examples set forth by the end user, where the synthesized program is configured to extract data from the log file and construct a table based upon the extracted data. With more particularly, technologies described herein relate to: 1) processing a log file to identify, without user input, header lines and comment lines in the log file, and subsequently filtering such lines so that they are not considered when a program is synthesized by way of PBE technologies; 2) identifying, without user input, boundaries of records in the log file, wherein PBE technologies can synthesize a program based upon the identified record boundaries and examples set forth by the end user, and further where the program is configured to construct a table based upon data extracted from the log file; and 3) responsive to receipt of user input with respect to a character or set of characters in the log file (when the user is selecting a substring for provision as an example to be used when synthesizing a program), setting forth a suggestion as to boundaries of the substring.

With reference to 1) noted above, the observation that regular data lines occur more often inside a log file when compared to how often header lines and/or comment lines occur in the log file is leveraged to learn a regular expression that can distinguish the header and comment lines in the log file from the regular data lines in the log file. For example, a model of the log file can be constructed, wherein the model is indicative of patterns in the log file (e.g., a majority of lines in the log file start with a first symbol, and presumably are data lines, while a minority of lines in the log file start with a second symbol, and therefore may be comment lines or header lines). Based upon the model of the log file, a regular expression can be learned from a relatively small predefined grammar, where the regular expression distinguishes the comment and header lines from the data lines. Further described herein are technologies related to ranking regular expressions when more than one regular expression is learned that can distinguish comment and header lines from regular data lines. In another example, header lines can be distinguished from regular data lines based upon the inference that header lines typically occur at the top of the log file. Moreover, in some cases, column names for an output table produced by way of PBE technologies may be inferred. More specifically, name delimiters can be inferred from a predefined set of delimiters to identify potential column names, and a similarity measure can be computed between the extracted column names and the type of the value in the respective output columns.

With respect to 2), responsive to header lines and comments lines being identified (and filtered), a regular expression (from the predefined grammar of regular expressions) can be learned to identify boundaries of records in the log file. More specifically, the regular expression can be learned by identifying common starting or ending patterns in the data lines of the log file.

With respect to the 3, user selection of a substring in the log file (when setting forth an example) by way of highlighting may be tricky, as precise starting and ending characters must be captured. Further, typing an entire substring is tedious, especially when substrings can be somewhat large. To address these issues, a substring in a record (that is to be provided to a PBE system as an example) can be inferred once the user has identified the starting characters. The inference of the substring can be based upon analysis of token boundaries around the starting characters, wherein the tokens come from a predefined set. The ending character of the substring often aligns with the ending of some token, such as date, number, lowercase characters, etc. Using such inference, suggestions can be provided to the user. For instance, the suggestion can be “snapping” a cursor to a suggested substring boundary responsive to the user selecting a starting character, thereby assisting the user in selecting the substring in the log file that the user intends to set forth as an example to the PBE system.

DETAILED DESCRIPTION

Described herein are various technologies pertaining to constructing tables based upon log files. With more particularity, technologies described herein relate to processing log files generated by computing devices, such that PBE technologies can be readily applied with respect to the processed log files. Log files are flat (non-hierarchical) files generated by computing devices. In an example, a computer-executable application can be configured to generate a log file that represents actions performed by the application (or by users of the application) over time. Log files typically include several lines of different types, where exemplary types of lines include header lines, comment lines, and data lines. Header lines often include data that is not well-suited for extraction from the log file and inclusion in a table. Such data can, for instance, identify an application that generated the log file, a time when the log file was constructed, and the like. Comment lines typically include user-generated content, often in the form of a text string. Again, information in comment lines is often not well-suited for inclusion in a table.

Data lines of a log file, however, include information that is often desirably extracted from the log file and placed into appropriate cells of a table. More specifically, the log file includes records, wherein each record includes at least a portion of a data line (although a record may include multiple data lines). A record of a log file includes a string, where the string comprises several substrings. In a non-limiting example, a record in a log file may include substrings pertaining to the following entities: process name, process ID, module path, symbol status, checksum, and time stamp. In many scenarios, it is desirable for the end user to identify a substring as being a field of a record, where the field and corresponding fields of other records in the log file are to be extracted from the log file and included in columns of an output table.

As indicated previously, the structure of log files (e.g., that log files typically include lines of various types) causes difficulties when PBE techniques are applied over log files. These difficulties are at least partially due to differences in structure between header lines, comment lines, and data lines in log files, and is further at least partially due to requiring an end-user to explicitly identify boundaries of records in log files. The technologies described herein pertain to processing log files, such that header lines and comment lines in the log file are identified without requiring user input. In other words, header lines and comment lines can be distinguished from data lines in the log file, such that a PBE system can skip over the header lines and comments lines when synthesizing a program based upon examples set forth by the user. Technologies described herein also pertain to identifying, without user input, record boundaries in the log file. This may include, for instance, determining that some records are portions of a single data line of a log file, some records include a single data line in the log file, and other records include multiple data lines in the log file.

Still further, described herein are technologies that pertain to setting forth suggestions to an end user when the end user identifies starting characters of a substring of a record. More specifically, when using PBE technologies, the user sets forth examples to a PBE system, wherein the examples can include an identification of a substring in a record of the log file. When the user identifies starting characters of the substring, the technologies described herein can automatically suggest an ending boundary of the substring. For instance, when the user highlights a character in a substring of a record, the highlighted region can be “snapped to” a most likely ending character of the substring.

With reference now toFIG. 1, an exemplary system100that facilitates processing a log file output by a computing device, such that the processed log file is well-suited for use in connection with PBE technologies, is illustrated. The system100is described as being included in an environment where data cleaning is to be performed, although the claimed technologies are applicable to other scenarios. For instance, an enterprise may include a plurality of server computing devices102-104, wherein data in log files generated by applications executing on the server computing devices102-104is desirably subject to analysis (e.g., it is desirable for client applications to produce abstractions over data in the log files, such as creation of visualizations of data in the log files). The system100further includes a data store106that is in communication with the server computing devices102-104. While the data store106is illustrated as being separate from the server computing devices102-104, it is to be understood that the data store106can be considered as a portion of at least one of the server computing devices102-104. A server computing device in the server computing devices102-104outputs a log file108, wherein the log file108is retained in the data store106. As indicated previously, the log file is a flat (non-hierarchical) file that can include lines of various types, wherein exemplary types of lines include header lines, comment lines, and data lines.

The system100further comprises a computing device110that is in communication with the data store106, such that the computing device110can access the log file108. The computing device110can be operated by a data cleaner111, who is tasked with constructing an output table based upon content in the data lines of the log file108, wherein the output table comprises cells that are populated based upon data extracted from the log file108. The computing device110includes at least one processor112and memory114that has a table generation system116loaded therein. Briefly, the table generation system116is configured to extract data from the log file108and construct a table118based upon the data extracted from the log file108. The table generation system116is configured to perform this task using PBE technologies. Thus, the data cleaner111identifies a portion of the log file108(e.g., a substring) that is to be extracted from the log file108, wherein a cell in the table118is to be populated based upon the portion of the log file108identified by the data cleaner111. That is, the table generation system116receives the substring identified by the data cleaner111, and synthesizes a program based upon the substring, wherein the program, upon receiving the log file108as input, outputs the table118, wherein the table118includes a column of cells that are populated with substrings that correspond to the substring identified by the data cleaner111.

Prior to synthesizing such program, however, the table generation system116processes the log file108such that the log file108is well-suited for PBE technologies. To that end, the table generation system116includes a record identifier component120that, prior to the data cleaner111setting forth examples, is configured to identify header lines and comment lines in the log file108and filter such lines from the log file108(without requiring the data cleaner111to explicitly distinguish comment and header lines from data lines in the log file108). The record identifier component120may utilize a variety of technologies in connection with distinguishing comment and header lines versus data lines in the log file108.

In a first example, the record identifier component120can load a portion of the log file108into the memory114(e.g., 200 lines of the log file108). The record identifier component120can construct a model of the portion of the log file108, and can identify a dominant pattern in the model. More specifically, since it is highly likely that there are several more data lines than there are comment or header lines, the record identifier component120can infer that lines in the portion of the log file108that conform to the dominant pattern are data lines, while other lines in the portion of the log file108loaded into the memory114(which do not conform to the dominant pattern) are comment or header lines. Thus, based upon patterns identified in the model of the portion of the log file108, the record identifier component120can learn a regular expression (from a relatively small predefined grammar) that distinguishes between the data lines in the portion of the log file108and comment and header lines in such portion of the log file108. The regular expression can be applied to the log file108, such that comment lines and header lines can be filtered from the log file108.

In some instances, the record identifier component120may learn multiple regular expressions that can distinguish between comment and header lines versus data lines. In such a case, the record identifier component120can rank the regular expressions according to a ranking metric. In an example, the ranking metric may be length of the regular expression, such that the shortest learned regular expression that can be used to distinguish between comment and header lines versus data lines in the log file108is ranked most highly. In another example, the metric may be number of occurrences of the regular expression in the portion of the log file108loaded into the memory114. For instance, a single break may separate consecutive data lines in the portion of the log file108, while multiple breaks may separate data lines from comment lines and header lines from data lines. Since consecutive line breaks will occur relatively infrequently in the portion of the log file108, the regular expression may be ranked more highly than a regular expression that is applicable a larger number of times in the portion of the log file.

The record identifier component120can further distinguish header lines from data lines based upon the knowledge that header lines tend to occur at the top of log files. Therefore, the record identifier component120can construct a model of the portion of the log file108, and determine a dominant pattern in the portion of the log file108(which presumably corresponds to data lines in the portion of the log file108). The record identifier component120may then start at the top of the portion of the log file108, and go line by line until a line in the portion of the log file108conforms to the dominant pattern is reached. The record identifier component120can identify the first k lines that do not conform to the dominant pattern as being header lines, and can filter such k lines from the log file108.

In some cases, the log file108will include a line that comprises potential column names in the output table118. For instance, the line can comprise delimiters that separate potential column names in the line. This line oftentimes conforms to the dominant pattern identified by the record identifier component120; however, substrings in this line appear different from corresponding substrings in other lines that conform to the dominant pattern. The record identifier component120can analyze the line, searching for delimiters from a predefined set of delimiters. When such line is identified, it can be inferred that substrings between the delimiters in this line represent potential column names in the output table118.

The record identifier component120, as referenced above, can further identify, without user input, boundaries of records in the log file108. To accomplish such task, the record identifier component120can analyze the portion of the log file108referenced above, and can build a model of the portion of the log file108. The model, for instance, can indicate that 193 lines start with a comma, while 7 lines do not start with a comma. This indicates that the lines that start with a comma correspond to separate records, and that a record is bounded at the front by a comma. In another example, the model of the portion of the log file108loaded into the memory114can indicate that 180 lines start with a timestamp, while 20 lines do not. The record identifier component120, based upon this model, can learn a regular expression from the aforementioned predefined grammar, where the regular expression indicates that each record is bounded at the front by a timestamp. Once the record identifier component120has identified, without user input, comment lines and header lines in the log file108, and has further identified, without user input, boundaries of records in the log file108, the table generation system116can utilize the processed log file108and can construct a table based upon examples set forth by the data cleaner111. It can be ascertained that these examples set forth by the data cleaner111need not include identification of record boundaries from the data cleaner111, and further need not include negative examples set forth by the data cleaner111(e.g., the data cleaner111need not explicitly identify header lines and comment lines in the log file108).

The table generation system116further includes an entry suggestion component122that is configured to assist the data cleaner111when the data cleaner111is providing examples to the table generation system116. More specifically, the computing device110includes an interface124(e.g., a keyboard, a soft keyboard, a touch-sensitive display, a mouse, etc.) that is employed by the data cleaner111to set forth examples to the table generation system116. As mentioned previously, each record in the log file108may include a string, wherein the string includes several substrings (which may be fields of the record). It is often the case that the data cleaner111desires that a column in the output table118corresponds to a substring in the record. Conventionally, the data cleaner111interacts with the interface124to set forth an example, wherein the example must precisely identify the substring that is to be extracted from the log file108. The data cleaner111, however, may accidentally select an incorrect portion of the string of the record, such that the table118output by the table generation system116does not reflect the intent of the data cleaner111. In another example, the data cleaner111may interact with the interface124to type the substring that is to be extracted from the log file108. When there is a typo in the information set forth by the data cleaner111to the table generation system116, the table118output by the table generation system116will not reflect the intent of the data cleaner111.

To cure these deficiencies, the entry suggestion component122is configured to set forth at least one suggestion to the data cleaner111based upon an initial character or set of characters identified by the data cleaner111in a string of the log file108. In a non-limiting example, when the data cleaner111highlights a character or characters in the string in the log file108, the entry suggestion component122can cause a cursor to “snap to” a suggested ending boundary of a substring, where the substring is bounded by the highlighted characters and the ending boundary. For instance, when a string includes a plurality of alphabetical characters followed by a whitespace, followed by a plurality of numerical characters, and the data cleaner111selects (e.g., by way of a mouse) a first alphabetical character in the alphabetical characters, the entry suggestion component122can cause a cursor to “snap to” the last alphabetical character in the alphabetical characters, such that the alphabetical characters are highlighted but the numerical characters are not highlighted. If the data cleaner111wishes to also include the numerical characters in the substring in an example that is to be set forth to the table generation system116, the data cleaner111can drag the mouse over a first character in the numerical characters and the entry suggestion component122can cause a cursor to “snap to” the last numerical character in the string. In another example, when the data cleaner111uses a keyboard to identify starting characters in a substring (e.g., by typing the starting characters in a text entry field), the entry suggestion component122can auto-populate the text-entry field with a remainder of a suggested substring or provide a list of suggestions from which the user can select one of the suggestions, thereby reducing data entry errors.

The entry suggestion component122, in connection with identifying suggestions, can tokenize the entire string of a record using a predefined set of tokens that represent characters of certain types, such that each token represents a respective character type (e.g., a first token can represent capital letters, a second token can represent lowercase letters, a third token can represent white space, a fourth token can represent numerical values, a fifth token can represent punctuation, etc.). Responsive to tokenizing the string of the record, and in response to receiving some input as to a start of a substring from the data cleaner111, the entry suggestion component122can suggest an ending boundary of the substring based upon token boundaries in the string in the record (e.g., wherein a token boundary is a transition between different tokens). For instance, a whitespace between a first plurality of alphabetical characters and a second plurality of alphabetical characters can indicate a potential substring boundary when an indication is received that the data cleaner111has selected a first character in the first plurality of alphabetical characters. In another example, a change from alphabetical characters (where each alphabetical character is represented by a first token) to numeric characters (where each numeric character is represented by a second token) can indicate a potential boundary when an indication is received that the data cleaner111has selected a first alphabetical character in the alphabetical characters. The entry suggestion component122can identify a prospective ending character of a substring by identifying the ending of some token, such as date, number, lowercase characters, etc. In addition, the entry suggestion component122may identify a plurality of suggestions and can rank such suggestions based upon any suitable metric. In an example, the entry suggestion component122can rank suggestions based upon selections that the data cleaner111has made in identifying other substrings being extracted from the log file108.

Now turning toFIG. 2, an exemplary schematic200that visually depicts processing of a log file202that can be performed by the record identifier component120is illustrated. As shown inFIG. 2, the log file202includes a header204, which may include a plurality of header lines, as well as a comment206, which may include a plurality of comment lines. The log file202further comprises a plurality of records208-212, wherein each record may include a string, and wherein each string may include a plurality of fields. For instance, the first record208includes a first string214, the second record210includes a second string216, and the nth record includes an nth string218. The record identifier component120, as noted above, without user input, identifies the header204and the comment206in the log file202. As indicated previously, the record identifier component120can construct a model of the log file202, identify a dominant pattern for lines in the log file202, and identify the first k lines that do not correspond to the dominant pattern as being the header204. Additionally or alternatively, the record identifier component120can learn a regular expression that distinguishes the header204and/or the comment206from the records208-212. For instance, comment lines in the comment206may start with a hash tag symbol, while other lines in the log file202start with a timestamp (e.g., where the dominant pattern is the timestamp beginning a line). The regular expression learned by the record identifier component120can distinguish between lines in the header204and/or the comment206from the data lines in the log file202.

Responsive to the record identifier component120identifying header lines and/or comment lines in the log file202, the record identifier component120can learn one or more regular expressions that are indicative of boundaries of the records208-212in the log file202. It is to be understood that it is not always the case that a new line in the log file202corresponds to a new record. For instance, a record of a log file may include multiple lines, and the regular expression learned by the record identifier component120can identify record boundaries that occur across multiple lines. The record identifier component120can output a processed log file214, such that when the table generation system116generates the table118based upon examples set forth by the data cleaner111, the header204and the comment206in the log file202are effectively skipped over by the table generation system116. Additionally, the table generation system116constructs the table118based upon the identified record boundaries.

With reference now toFIG. 3, an exemplary schematic300that depicts operation of the entry suggestion component of122is illustrated. As indicated previously, the first record208of the log file202can include the first string214, and the string can include several fields (where a field is a substring of the string214that is to be extracted from the record214). Boundaries between fields may not be readily identifiable without explicit user input. Conventionally, the data cleaner111is required to precisely identify the desired substring in the first string214, where the data cleaner111may incorrectly identify a field (e.g., the data cleaner111may mis-click). As indicated previously, the entry suggestion component122can tokenize the string in the record208, where potential field boundaries can be indicated by token boundaries. Thus, in this example, when the data cleaner111highlights characters corresponding to the token sequence “##” (e.g., the beginning of a field), the entry suggestion component122can compute a plurality of suggestions for the end of the field. For instance, a cursor can be “snapped to” the character corresponding to the token boundary between the “#” token and the “C” token. The entry suggestion component122can determine several suggestions302, rank the suggestions, and select a suggestion based upon the ranking (e.g., the entry suggestion component122can rank the suggestions based upon previous substrings extracted from the log file or other log files by the data cleaner111, by people in the same department as the data cleaner111, etc.).

Now referring toFIG. 4, another schematic400illustrating operation of the record identifier component120with respect to an exemplary log file402is illustrated. In the exemplary log file402, a header of the log file402is included in the first six lines of the log file402. The record identifier component120can construct a model of the log file402, which indicates that a relatively large number of lines start with a comma, while a relatively small number of lines do not. Thus, the dominant pattern identified by the record identifier component120is a comma at the beginning of a line. When identifying the header, the record identifier component120can start at the top of the log file402and go line by line downward until a line having the dominant pattern is reached (which can inferred to be a line other than a header line). In this case, the record identifier component120can identify that the top six lines do not conform to the dominant pattern, and can therefore identify such lines as header lines in the log file402.

It can also be ascertained that the seventh line in the log file402includes delimiters that separate potential column names in the output table118. The seventh line, however, is clearly distinguishable from the lines that follow it. The record identifier component120can determine, for instance, that length of the seventh line is somewhat shorter than all lines that follow it, and can infer that the seventh line includes delimiters that separates text that corresponds to column names. In another example, the record identifier component120can analyze text in the seventh line and can compare the text with other lines in the log file402, and can infer that the seventh line in the log file402includes delimiters (e.g., commas) from amongst a plurality of predefined delimiters. The record identifier component120outputs a processed log file404, where the header lines and the line that include the potential column names are shown as being struck through, such that the table generation system116skips those lines when extracting data from the log file402for inclusion in the table118(based upon example set forth by the data cleaner111).

Now referring toFIG. 5, another exemplary schematic500that illustrates operation of the record identifier component120is illustrated. A log file502includes a plurality of data lines. In this example, the record identifier component120identifies, without input from the data cleaner111, boundaries of records in the log file502. The record identifier component122constructs a model of the log file502, wherein the model can indicate that the majority of lines in the log file502start with a timestamp. Lines7,8, and9in the log file502, however, fail to include a timestamp. Since the record identifier component120has already determined that lines7-9of the log file502are not header lines or comment lines (e.g., the record identifier component120identifies record boundaries subsequent to identifying header and comment lines), the record identifier component120can learn a regular expression that delineates boundaries of records in the log file502. In this example, the regular expression can correspond to a timestamp. Thus, lines6-9in the log file502all belong to one record (as shown in a processed log file504), while other records include a single line. The record identifier component120can output a processed log file502, where boundaries of records are delineated.

FIGS. 6-8illustrate exemplary methodologies relating to processing a log file in connection with constructing a table based upon data extracted from the log file. While the methodologies are shown and described as being a series of acts that are performed in a sequence, it is to be understood and appreciated that the methodologies are not limited by the order of the sequence. For example, some acts can occur in a different order than what is described herein. In addition, an act can occur concurrently with another act. Further, in some instances, not all acts may be required to implement a methodology described herein.

Now referring solely toFIG. 6, an exemplary methodology600for constructing a table based upon automatically identified record boundaries is illustrated. The methodology600starts at602and, at604, a portion of a log file is loaded into memory of a computing device. At606, without user input, boundaries of records in the log file are identified. As described above, these boundaries can be identified responsive to header lines and comment lines in the log file being distinguished from data lines in the log file, and can further be identified by learning a regular expression that is indicative of record boundaries in the log file. At608, a table is constructed based upon the identified record boundaries, wherein the table comprises rows that correspond to the identified records. Such tables, as noted above, are also constructed based upon fields in the log file identified by the data cleaner111(and provided as examples to the table generation system116). The methodology600completes at610.

Now referring toFIG. 7, an exemplary methodology700that facilitates constructing a table based upon identified comment lines and header lines in a log file is illustrated. The methodology700starts at702, and at704, a portion of a log file is loaded into memory of a computing device. At706, without user input, comment lines and header lines are identified in the portion of the log file. As described above, a model of the log file can be constructed, and header lines can be identified as being the top k lines in the log file that do not conform to a dominant pattern in the model. Additionally or alternatively, a regular expression can be learned that distinguishes header lines and comment lines from data lines in the log file. At708, a table is constructed based upon the identified comment lines and header lines, wherein the table fails to include text in the comment rows and header rows, and further wherein the table includes rows that correspond to records in the log file. The methodology700completes at710.

Turning now toFIG. 8, an exemplary methodology800that facilitates assisting a data cleaner in identifying a field that is to be extracted from a record in the log file is illustrated. The methodology800starts at802and, at804, it is detected that a user has selected at least one character in a record of a log file. The at least one character can be inferred to be the beginning character of a field (substring) that is desirably extracted by a data cleaner. At806, a suggestion is caused to be presented on a display responsive to detecting that the data cleaner has selected the at least one character. For instance, the suggestion may be: 1) a “snap to” a potential field boundary; 2) auto-populating a text entry field corresponding to a potential field boundary, etc. At808, an indication is received that the user has selected the suggestion (confirming the suggested field boundary) and, at810a table is constructed based upon the user selecting the suggestion (e.g., the selected field is provided to the table generation system116as an example, and a program is learned that extracts corresponding fields in other records of the log file based upon such example). The methodology800completes at812.

Referring now toFIG. 9, a high-level illustration of an exemplary computing device900that can be used in accordance with the systems and methodologies disclosed herein is illustrated. For instance, the computing device900may be used in a system that distinguished header lines and comments lines in a log file from data lines in the log file. By way of another example, the computing device900can be used in a system that is configured to identify record boundaries in a log file. The computing device900includes at least one processor902that executes instructions that are stored in a memory904. The instructions may be, for instance, instructions for implementing functionality described as being carried out by one or more components discussed above or instructions for implementing one or more of the methods described above. The processor902may access the memory904by way of a system bus906. In addition to storing executable instructions, the memory904may also store log files, tables, etc.

The computing device900additionally includes a data store908that is accessible by the processor902by way of the system bus906. The data store908may include executable instructions, log files, output tables, etc. The computing device900also includes an input interface910that allows external devices to communicate with the computing device900. For instance, the input interface910may be used to receive instructions from an external computer device, from a user, etc. The computing device900also includes an output interface912that interfaces the computing device900with one or more external devices. For example, the computing device900may display text, images, etc. by way of the output interface912.