METHODS AND SYSTEMS FOR CREATING, STORING, AND MAINTAINING CUSTODIAN-BASED DATA

The present disclosure is directed to systems and methods for managing custodian-based data. The method includes, for example, (i) retrieving custodian-based data associated with multiple custodians; (ii) analyzing metadata items associated with the custodian-based data, and the metadata items includes one or more custodian actions; (iii) generating immutable identifiers for the custodian-based data associated with the custodian actions; and (iv) storing the custodian-based data in a raw data form. The method also enables a query of the custodian-based data based on the custodian actions.

TECHNICAL FIELD

The present technology is directed to systems and methods for custodian-based data. More particularly, systems and methods for creating, managing, storing, maintaining, and querying custodian-based email data are disclosed herein.

BACKGROUND

Custodian-based data, such as email data, is an important source of information in modern life. For example, the custodian-based data can be used as evidence in litigation. To be able to show that a custodian is aware of and/or whether they have taken actions to obfuscate certain information, the actions of the custodian must be recorded or stored. It can be challenging for traditional data management systems to effectively record or store such custodian actions. Therefore, there is a need and it is advantageous to have an improved method and system to address the foregoing issue.

DETAILED DESCRIPTION

The present technology is directed to systems and methods for creating, managing, storing, maintaining, and querying custodian-based data. In some embodiments, the custodian-based data can include emails, messages, account information, transaction histories, etc. Traditional approaches of managing custodian-based data include storing such data in a file with corresponding metadata. For example, an email can be saved in EML format (also known as “RFC-822” file format), and can be accessed via Microsoft Outlook or Apple Mail. In some embodiments, emails in EML format can include attachments encoded therein in text format.

The metadata of an email in EML format indicates the subject, sender, recipients and date of the email. Such metadata does not provide sufficient information regarding how the email has been accessed, processed, or handled by its custodian (i.e., “custodian actions”) after the custodian receives the email. For example, after the custodian accesses an email, the custodian may try to achieve, delete, and/or mark as “unread” that email. In other examples, the custodian may try to assign a flag identifier (e.g., “confidential,” “urgent,” “to be deleted,” “important,” “to be ignored,” etc.) to the email. Traditional metadata of an email (such as in EML format) does not provide information regarding the foregoing custodian actions.

To address this need, the present disclosure provides systems and method for managing custodian-based data and enables an operator to analyze, store, and/or search the data effectively and efficiently. Generally speaking, for each custodian action performed (or some actions of interests, depending on user preference), the present method can identify the custodian action, generate immutable identifiers, and associate them with the custodian-based data. The immutable identifiers can be generated and stored in a practically “real-time” manner. For example, in some embodiment, the immutable identifiers can be generated once per 6, 12, or 24 hours. The generated immutable identifiers record all the identified custodian actions during this time period. The sooner the immutable identifiers are generated after the custodian actions were performed, the less likelihood that the custodian-based data is altered, tampered with, or compromised.

The present method can then store the custodian-based data with the generated immutable identifiers. By this arrangement, the present method effectively preserves and stores the custodian-based data such that it can be searched, queried, and/or analyzed at a later time (e.g., evidence for litigation).

One aspect of the present technology includes enabling an operator to retrieve and store custodian-based data by recording custodian actions that have been performed on the custodian-based data. In some embodiments, the present method includes, for example, (i) retrieving custodian-based data associated with multiple custodians; (ii) analyzing metadata items (which include one or more custodian actions) associated with the custodian-based data, (iii) generating immutable identifiers for the custodian-based data associated with the custodian actions; (iv) generating immutable identifiers for the custodian-based data associated with the custodian actions; and (v) storing the custodian-based data in a raw data form (e.g., in binary form).

Another aspect of the present technology includes enabling an operator to query or search custodian-based data based on custodian actions. For example, the operator can search emails from a sender that were read and later marked as “unread” during a certain period of time. In this example, the custodian action can be “accessing an email and later marking it as unread.” In some embodiments, the custodian actions can be defined based on user preferences.

FIG. 1is a schematic diagram illustrating a system100in accordance with embodiments of the present technology. As shown inFIG. 1, the system100includes a computing device101, a source data server103, and a target data server105. The computing device101is configured to (i) retrieve custodian-based data from the source data server103, (ii) process the custodian-based data, and (iii) store the processed custodian-based data in the target data server105. In some embodiments, the computing device101can query or search the processed custodian-based data stored in the target data server105. In some embodiments, the computing device101can include one or more processors and memories configured for implementing the foregoing tasks. Embodiments of the computing device101are discussed in detail with reference toFIG. 4.

Suitable systems and methods for searching processed custodian-based data are further described in co-pending U.S. patent application Ser. No. ______, filed ______, and entitled METHODS AND SYSTEMS FOR CUSTODIAN-BASED SEARCHING, (attorney docket no. 136566-8002.US00) and co-pending U.S. patent application Ser. No. ______, filed ______, and entitled METHODS AND SYSTEMS FOR CUSTODIAN BASED HYDRATION, (attorney docket no. 136566-8003.US00), the disclosures of which are incorporated herein by reference in their entireties.

In some embodiments, the source data server103can include an email server, a local/cloud server, and/or other suitable devices that store custodian-based data to be retrieved by the computing device101. The computing device101can first communicate with the source data server103to learn what custodian-based data (e.g., emails of employees in Company X) are stored therein and its format (EML files) (e.g., Step11shown inFIG. 1). The source data server103includes an activity log recording all activities or actions associated with the custodian-based data. The activities and actions can include actions by a custodian (“custodian actions”) performed on the custodian-based data. For example, the custodian-based data can include an email. The custodian of the email can be a sender, a direct recipient, and/or an indirect recipient (e.g., “carbon copied” or “blind carbon copied”). Examples of the custodian actions of the email include deleting the email, archiving the email, assigning a flag identifier to the email (e.g., showing status of the email such as confidential, urgent, to be deleted, important, and/or to be ignored), and attempting to change the status of the email (e.g., marking the email as “unread” after accessing the email).

The computing device101can then create an immutable identifier107for each of the actions or activities in the activity log in the source data server103. In some embodiments, the immutable identifiers107can be generated by an application implemented in the source data server103. The computing device101then causes the custodian-based data and the immutable identifiers107to be stored in the target data server105(e.g., Step13shown inFIG. 1).

As shown inFIG. 1, the custodian-based data stored in the target data server105includes a metadata portion109and a raw data portion111. The metadata portion109is indicative of a custodian1091, a custodian action1092, and time1093that the custodian action was performed. The raw data portion111can include the content of the custodian-based data and can be in binary form (e.g., to save storage space). The metadata portion109and the raw data portion111are associated with the immutable identifier107, such that the computing device101can query or search the custodian-based data stored in the target data server105based on the metadata portion109(e.g., a search using “custodian,” “custodian action,” and/or “time” as keywords) (e.g., Step15shown inFIG. 1). By this arrangement, the system100enables an operator to effectively manage, store, and query the custodian data from the source data server103.

FIG. 2is a schematic diagram illustrating another system200in accordance with embodiments of the present technology. Similar to the system100described inFIG. 1, the system200includes a computing device201and an email data server203. The system can have (i) a query server205configured to handle queries/searches and (ii) a database207configured to store raw data.

The computing device201can first communicate with the email data server203and analyze the email data stored therein (e.g., Step21shown inFIG. 2). The email data server203includes an activity log recording all activities or actions associated with the email data. The activities and actions can include actions by a custodian (“custodian actions”) performed on the email data. The custodian of the email can be a sender, a direct recipient, and/or an indirect recipient (e.g., “carbon copied” or “blind carbon copied”). Examples of the custodian actions of the email include deleting the email, achieving the email, assigning a flag identifier to the email (e.g., showing status of the email such as confidential, urgent, to be deleted, important, and/or to be ignored), and attempting to change the status of the email (e.g., marking the email as “unread” after accessing the email).

The computing device201can generate an immutable identifier for each of the actions or activities in the activity log in the email data server203. In some embodiments, the immutable identifiers can be generated by an application implemented in the email data server203. The computing device201can then generate metadata (e.g., the metadata portion109discussed above inFIG. 1) for each email of the email data based on the custodian actions, and associate the metadata with the immutable identifier. For example, the metadata can indicate a custodian action, a corresponding custodian, and/or the time that the custodian action was performed. The immutable identifiers and the metadata can be stored in the query server205(e.g., Step23shown inFIG. 2). The computing device201can store the immutable identifiers and the content of the emails of the email data (e.g., the raw data portion111discussed above inFIG. 1) in the database207(e.g., Step24shown inFIG. 2).

Based on the immutable identifiers, the system200enables an operator to search or query the email data in the query server205(e.g., Step25shown inFIG. 2). The query server205can pull the content of the emails from the database207based on the immutable identifiers (e.g., Step27shown inFIG. 2), if the operator requests doing so.

In the illustrated embodiments, the computing device201, the query server203, and the database207can each be implemented as a distributed system across more than one devices connected via a network.

FIG. 3is a schematic diagram illustrating a data structure of a custodian-based data set300in accordance with embodiments of the present technology. As shown, the custodian-based data set300includes immutable identifiers301, a metadata portion303, and a data portion305. The immutable identifiers301are configured to associate the metadata portion303and the data portion305such that the data portion305can be searched or queried based on the metadata portion303. In some embodiments, the immutable identifiers301can include a serial number, a string, a symbol, an object, a link, and/or other suitable identifiers. The data portion305can include email data such as EML files3051and corresponding attachments3052. The data portion305can be in binary form.

In some embodiments, the metadata portion303can be a JavaScript Object Notation (JSON) message. JSON is a lightweight, text format that is language independent. JSON messages are easy for humans to read and write as well as for machines to parse and generate. The metadata portion303can indicate a custodian section3031, an application section3032, an action section3033, and a time section3034. The custodian section3031indicates a custodian of a data piece (e.g., an email, a message, etc.) of the data portion305. The application section3032indicates an application (e.g., Microsoft Outlook) that was used to access the data piece. The action section3033indicates a custodian action that was performed to the data piece. The time section3034indicates the time that the custodian action was performed.

The immutable identifiers301are associated with the sections3031-3034such that an operator can search or query the data portion305based on these sections3031-3034. For example, the operator can search all the custodian actions performed by custodian C1using Application A1during time period T1. As another example, the operator can search all data pieces that were “marked as unread” by custodian C2using application A2during time period T2. By this arrangement, the present technology provides a data structure to store/maintain and search/query the custodian-based data in an efficient and convenient fashion.

FIG. 4is a schematic diagram illustrating components in a computing device (e.g., a client device, a server, etc.) in accordance with embodiments of the present technology. The computing device400can be implemented as a server, a client device, a distributed computing system, and/or other suitable devices. Examples of the computing device400include the computing devices100,200inFIGS. 1 and 2. The computing device400is configured to process the methods (e.g.,FIGS. 5-7) discussed herein. The illustrated computing device400is only an example of a suitable computing device and is not intended to suggest any limitation as to the scope of use or functionality. Other well-known computing systems, environments, and/or configurations that may be suitable for use include, but are not limited to, personal computers (PCs), server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics such as smart phones, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

In its basic configuration, the computing device400includes at least one processing unit402and a memory404. Depending on the exact configuration and the type of computing device, the memory404may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two. This basic configuration is illustrated inFIG. 4by dashed line406. Further, the computing device400may also include storage devices (a removable storage408and/or a non-removable storage410) including, but not limited to, magnetic or optical disks or tape. Similarly, the computing device400can have an input device414such as keyboard, mouse, pen, voice input, etc. and/or an output device416such as a display, speakers, printer, etc. Also included in the computing device400can be one or more communication components412, such as components for connecting via LAN, WAN, point to point, any other suitable interface, etc.

The computing device400can include a data management/query module418configured to implement methods for managing and querying custodian-based data. The data management/query module418is configured to receive and analyze custodian-based data, store/manage the analyzed custodian-based data, and search the stored custodian-based data. In some embodiments, the data management/query module418can be in form of instructions, software, firmware, as well as a tangible device.

The computing device400includes at least some form of computer readable media. The computer readable media can be any available media that can be accessed by the processing unit402. By way of example, the computer readable media can include computer storage media and communication media. The computer storage media can include volatile and nonvolatile, removable and non-removable media (e.g., removable storage408and non-removable storage410) implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The computer storage media can include, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other tangible medium which can be used to store the desired information.

FIG. 5is a flow diagram showing a method500in accordance with embodiments of the present technology. The method500can be implemented by a computing device (e.g., the computing device101,201, or400) or any other suitable devices. The method500starts at block501by retrieving custodian-based data associated with multiple custodians. In some embodiments, the custodian-based data can include email data. The multiple custodians can include email users, each of which can be identified by an email address (e.g., XYZ@ABC.com).

At block503, the method500continues by analyzing metadata items associated with the custodian-based data. The metadata items can include a custodian, one or more custodian actions, and time that the one or more custodian actions were performed.

At block505, the method500continues by generating immutable identifiers for the custodian-based data associated with the custodian actions. In some embodiments, for each custodian action, there can be a corresponding immutable identifier. For example, an immutable identifier “ABC-XYZ-20210101-0650PM-ACTION-A1” can be generated for action “A1” performed by custodian “XYZ” of Company “ABC” to a data piece of the custodian-based data at “6:50 p.m.” on “Jan. 1, 2021.” In other embodiments, the immutable identifiers can be in various forms. At block507, the method500continues to store the custodian-based data in a raw data form. For example, the custodian-based data can be stored in binary form. Storing the custodian-based data in binary form can reduce storage space and accordingly enhance an overall efficiency.

In some embodiments, the method500can include enabling a query of the custodian-based data based on the custodian actions. In some embodiments, the custodian-based data can include email data. In some embodiments, the email data can include information in JSON format, information in EML format, an attachment to an email, and/or a link in an email. The multiple custodians can include a sender of an email in the email data and/or a recipient of the email.

In certain examples, the metadata items can include a sender of an email in the email data, a direct recipient of the email, an indirect recipient of the email, a flag identifier of the email, time that the one or more custodian actions are performed to the email.

In some instances, the custodian actions can include (i) deleting an email of the email data; (ii) achieving an email of the email data; (iii) assigning a flag identifier to an email of the email data; and/or (iii) marking an email in the email data as unread after the email is accessed. The flag identifier can be indicative of one or more following statuses of the email: confidential, urgent, to be deleted, important, and/or to be ignored.

FIG. 6is a flow diagram showing a method600in accordance with embodiments of the present technology. The method600can be implemented by a computing device (e.g., the computing device101,201, or400) or any other suitable devices. The method600starts at block601by retrieving a list of multiple custodians. At block603, the method600continues by retrieving custodian-based data associated with the multiple custodians. At block605, the method600continues to analyze metadata items associated with the custodian-based data. The metadata items can include one or more custodian actions.

At block607, the method600continues by generating immutable identifiers for the custodian-based data associated with the custodian actions. At block609, metadata is generated for the custodian-based data corresponding to the immutable identifiers. For example, for each custodian action, an immutable identifier can be generated. At block611, the method600includes identifying an attachment associated with an email of the custodian-based email data. At block613, the method600continues by storing the custodian-based data and the attachment in a raw data form.

In some embodiments, the method600further includes enabling a query of the custodian-based data based on the custodian actions. In some embodiments, the method600further includes (i) retrieving the custodian-based data associated with the multiple custodians in a real-time manner; (ii) verifying whether the attachment associated with the email is included in the custodian-based email data; and/or (iii) in an event that the attachment associated with in the email is not included in the custodian-based email data, retrieving the attachment via a link in the email.

FIG. 7is a flow diagram showing a method700in accordance with embodiments of the present technology. The method700can be implemented by a computing device (e.g., the computing device101,201, or400) or any other suitable devices. At block701, a user list is received. In some embodiments, the user list can include multiple names, account names, titles, email addresses, and/or other suitable information.

At block703, information regarding “Folders Manifest from an email box” can be retrieved. In some embodiments, “Folders Manifest” can be a text list of file or folder contents of the email box. The information regarding “Folders Manifest” can indicate the number and types of folders that an email account may have. For example, an email account can have a “to be deleted” folder, a “draft” folder, an “important folder,” “to be processed” folder, etc. In some embodiments, the information regarding “Folders Manifest” can be in JSON format.

At block705, by analyzing the information regarding “Folders Manifest,” immutable identifiers are generated and assigned to actions or items in each folder. At block707, metadata associated with the immutable identifiers can be generated (e.g., in JSON format, noted as “New JSON messages by Immutable IDs” at block707. In some embodiments, if an attachment to an email is in text format, it can also be included in the JSON message.

At block709, the method700continues to pull email content (e.g., EML files) based on the generated immutable identifiers. For example, an immutable identifier “ABC-XYZ-19970505-0343AM-UNREAD-A2” can be generated for action “A2” that the custodian “XYZ” of Company “ABC” marked an email as “unread” at “3:43 a.m.” on “May 5, 1997.” The custodian's action was recorded by moving the email from folder “Inbox” to “unread” folder. Based on the immutable identifier corresponding to that email, an EML file of that email can be pulled and stored.

At decision block711, the method700determines whether an attachment associated with the email is already present or pulled. If affirmative, the process moves to block713. If negative, the process moves to block715to individually download that attachment.

At decision block713, the method700determines whether there is a “modern attachment” associated with the email. The term “modern attachment” refers to a link included in the email and directed to a remote network address or location. For example, a link to a file saved in a cloud server. If affirmative, the process moves to block717to download or pull the file indicated by the modern attachment. If negative, the process then returns for further process.