ASSISTIVE ASSESSMENT PLATFORM

Various embodiments are directed to an assistive assessment platform directed to improving the efficiency of student assessment administration, the ease of monitoring student progress, evaluator accountability, and the scoring of education outcomes, such as, for example, in pre-kindergarten (pre-K) to twelfth grade environments. The assistive assessment platform described herein allows for the simultaneous remote evaluation of a single student by an arbitrary number of authenticated teachers.

TECHNICAL FIELD

The present disclosure generally relates to the tools, techniques, and platforms used by teachers to create or administer student educational assessments.

BACKGROUND

Educational assessments have become an increasingly prevalent tool used to understand, refine, and improve learning opportunities for students. Educational assessments are systematic processes for documenting and utilizing data to establish measurable learning outcomes, gathering information to analyze and interpret to determine whether student learning matches expectations, and using the information to improve learning opportunities. However, as the number of student assessments and types of assessments available for students has increased, coupled with the increased use of educational technology for assessment in the classroom may be burdensome to administer and manage the educational assessments.

The detailed description is set forth with reference to the accompanying drawings. The drawings are provided for purposes of illustration only and merely depict example embodiments of the disclosure. The drawings are provided to facilitate understanding of the disclosure and shall not be deemed to limit the breadth, scope, or applicability of the disclosure. The use of the same reference numerals indicates similar but not necessarily the same or identical components; different reference numerals may be used to identify similar components as well. Various embodiments may utilize elements or components other than those illustrated in the drawings, and some elements and/or components may not be present in various embodiments. The use of singular terminology to describe a component or element may, depending on the context, encompass a plural number of such components or elements and vice versa.

DETAILED DESCRIPTION

This disclosure relates to, among other things, systems, methods, computer-readable media, techniques, and methodology for an assistive assessment platform. The assistive assessment platform, which may also be known as the teacher assistive scoring platform (TASP), is directed to increase the efficiency of assessment administration, monitoring, and scoring of education outcomes, such as, for example, in pre-kindergarten (pre-K) to twelfth grade environments. Additionally, the assessments provided to student devices as described herein may be adaptive assessments that may change in real-time while a student is taking an assessment. For example, the assessment may adapt based on the responses being provided by the student to a student device (as well as other factors, as will be described in further detail below). That is, the assessment stimulus being presented to the student on the student device may not be fixed, but rather may be selected from a bank of available assessment stimulus based on the responses being provided by the student to previous assessment tasks. In this regard, assessments may not necessarily be the same for every student, and assessment stimulus that are likely more representative of a student's ability level may be provided to the student in real-time. Additionally, the assessment may continue to identify further assessment stimulus to provide to the student device until it is determined that a confidence value that a student ability metric produced based on the responses provided to the student device is within a given threshold. The manner in which these real-time updates are performed may be described in additional detail with respect toFIG. 6, for example. It should be noted that reference may be made herein to an “assessment piloting platform,” and an “assistive scoring platform” (or like terms). The assessment piloting platform may refer to a platform responsible for managing rules and conditions for assessment administration, whereas the assistive scoring platform may refer to the use of a student device and a proctor or teacher device (the terms “proctor” and “teacher” may be used interchangeably herein) to present and score assessments as described herein. However, in some cases, the terms may be used interchangeably as well. Additionally, an “assessment stimulus” may refer to an item (or a portion of a item, such as a question or prompt included within a task) that may be presented to a student device during an assessment (for example, as depicted inFIG. 8).

Some educational assessments may be inefficient and have inaccurate scoring of a student's accuracy in response to a presented stimulus. For example, in grades pre-K through grade5, reading assessment systems frequently rely on the oral response of a student to a presented stimulus, such as stating the name or sound of a letter or reading a non-sense word. Math assessment systems in the same grade may utilize verbal responses for number recognition and math computation skills to assess ability. For middle and high school grade levels, reading and math assessment systems may increasingly rely on computer adaptive technologies, classroom discourse, and observations to understand performance in domains. Currently available educational technologies may be limited as technology-based scoring systems are not able to allow one or more teachers to independently assess student performance using an independent scoring device from the student assessment experience. Common methods for scoring verbal responses include the use of paper and pencil assessments or shared-device technologies. Paper and pencil assessments may require the student and teacher to use separate printed forms, whereby the student provides a verbal response to the presented stimuli on the paper and the teacher concurrently scores student performance on their respective printed form. The limitation of such an approach is that assessments must be printed, administered by the teacher and student separately, hand scored by the teacher, and then entered for each student by the teacher into a local or web-based assessment-entry platform. Shared device technologies exist, whereby assessments may display a stimulus on the screen, the student provides a response, and the teacher scores the item on the same screen the student using. However, this may require the teacher to pass a device back and forth to the student when a tablet or mobile device is used, or the teacher must actively score using dummy-coded keystrokes (e.g., “Z” on a keyboard is used for correct and “M” on a keyboard is used for incorrect) when a laptop or PC is used. This may be problematic because if the teacher and student are sharing a device, the student may be able to identify when the teacher is marking a response as correct and incorrect, which may bias future responses by the student. Requiring dummy-coded keystrokes may also place a cognitive burden on the teacher, who may be required to remember the dummy-coded keystroke while also trying to accurately score a student's oral response. Although a shared-technology platform allows for simplicity in scoring, it is cumbersome for scoring and does not allow for multiple-teacher observations or evaluation of student performance. The systems and methods described herein are directed to creating a shared space where a student and one or more teachers share an assessment session, allowing the student to participate in an educational assessment and the teacher(s) to monitor and score the assessment on a separate device in real or near real time.

The systems and methods described herein are directed to an assistive assessment platform for increasing the efficiency of assessment scoring in education environments. In some embodiments, the assistive assessment platform may create a shared assessment session that a student and one or more teachers can simultaneously engage in. As the administration of educational assessments become more technology-based, issues arise that are associated with the administration of student assessments and teacher scoring of the assessments. The assistive assessment platform may bridge the gap between student performance and teacher evaluation by synchronizing the device-based experience for both students and teachers. In some embodiments, the assistive assessment platform may include different elements, such as a teacher scoring experience (e.g., via a teacher device), a student assessment experience (e.g., via a student device), and/or an assistive assessment server, which may also be referred to as a teacher assistive scoring platform communication server. The teacher scoring experience (TSE) may be a web-based presentation for creating shared assessment sessions, recording student responses, and/or scoring the student responses as shown in the shared assessment session. The student assessment experience (SAE) may be a web-based presentation for assessments and item delivery of teacher-created shared assessment sessions. The assistive assessment server may be a virtual machine or a cloud-based hosting solution where information is passed between the shared sessions of the TSE and the SAE. The server may also facilitate communication between the assessment data and any third-party learning management system or learning record stores.

In some embodiments, the assistive assessment platform may create a shared session for administering one or more assessments. In an example embodiment, a teacher may launch an application (e.g., web browser or dedicated application) on a teacher device and may provide authentication credentials. The assistive assessment server may communicate with the teacher device and authenticate the teacher and/or teacher device using the authentication credentials. The teacher may be presented with various assessments that are specifically designed for multi-device proctoring. In some embodiments, the teacher may select an assessment from the presented assessments. The teacher may select a student from a list or otherwise provide student information (e.g., name, class, grade, student ID, etc.). The web browser or dedicated client of the teacher device may create and transmit a request to the assistive assessment server. The request may indicate a selection of an assessment, student information, and the like.

The assistive assessment server may receive the request and may create an assessment lobby. The assessment lobby may be the mechanism by which the teacher device and the student device can share data and be synchronized for the administration of the assessment. The assessment lobby may include at least two properties—an assessment lobby invite code and an assessment lobby token. The assessment lobby invite code may be used by a teacher device or a student device to join the assessment lobby. The assessment lobby token may be used for secure communication between the teacher device and/or the student device with the assistive assessment server. In some embodiments, the assessment lobby invite code may be transmitted to the requesting teacher device. The assessment lobby invite code may be shared with the designated student. The invite code may also be shared with other authenticated teachers who, for example, wish to join the assessment lobby and observe the administration of the selected assessment.

In some embodiments, the student identified to take the assessment may receive the invite code from the teacher. In some embodiments, the teacher may share the code with the student. The student may interact with a student device by launching a web browser or dedicated client of the student device. The student may enter the invite code into the student device. The student device may generate a request to join the assessment lobby identified by the invite code. The assistive assessment server may determine the validity of the invite code. The assistive assessment server may send the student device the assessment lobby token associated with the assessment lobby. The student device may use the assessment lobby token to join the assessment lobby, enabling the student device and the teacher device to be synchronized to the same shared session (e.g., assessment lobby) hosted by the assistive assessment server.

Various illustrative embodiments have been discussed above. These and other example embodiments of the disclosure will be described in more detail hereinafter through reference to the accompanying drawings. The drawings and the corresponding description are provided merely for illustration and are not intended to limit the disclosure in any way. It should be appreciated that numerous other embodiments, variations, and so forth are within the scope of this disclosure.

Illustrative Use Cases and System Architecture

FIG. 1depicts an illustrative data flow between various components of an illustrative system architecture100for an assistive assessment platform in accordance with one or more embodiments of the disclosure. One or more illustrative teacher device(s)104A operable by one or more teacher(s)102A and student device(s)104B operable by one or more student(s)102B are illustratively depicted inFIG. 1. The teacher device(s)104A and student device(s)104B may include any suitable processor-driven computing device including, but not limited to, a desktop computing device, a laptop computing device, a server, a smartphone, a tablet, and so forth. For ease of explanation, the teacher device(s)104A and student device(s)104B (collectively user device(s)104) may be described herein in the singular; however, it should be appreciated that multiple user device(s)104may be provided.

In an illustrative embodiment, a teacher of the one or more teacher(s)102A may interact with a teacher device104A to launch an application (e.g., web browser or dedicated client application) select an assessment from a list of available assessments. The teacher102A may interact with the teacher device104A to select an assessment and enter information associated with a student102B to whom the assessment is to be administered. The teacher device104A may generate a request for a shared session (e.g. assessment lobby) that includes the selection of the assessment and student information provided by the teacher102A. The teacher device104A may transmit the request to the assistive assessment server106.

The assistive assessment server106may generate a shared session based on the information provided by the teacher device104A. The assistive assessment server106may generate shared session information, such as a shared session invite code and a shared session token. The shared session invite code may be used by a teacher device104A or a student device104B to join the shared session. The shared session token may be used for secure communication between the teacher device104A and/or the student device104B with the assistive assessment server106. The assistive assessment server106may transmit the shared session information to the teacher device104A. The teacher device104A may retain the shared session token and use the token to join the shared session.

The teacher102A may share the shared session invite code with a student102B. The teacher102A may facilitate transmission of the shared session invite code from the teacher device104A to the student device104B. In some embodiments, the teacher102A may share the code with the student102B who may then enter the code using an application on the student device104B. The student device104B may transmit a request to join the shared session to the assistive assessment server106. The request may include the shared session invite code. The assistive assessment server106may validate the student device104B based on the shared session invite code and may transmit the shared session token associated with the shared session to the student device104B. The student device104B may use the shared session token to join the shared session. Likewise, the teacher device104A may use the shared session token to join the shared session. The student102B and teacher102A may interact with their respective devices to complete the assessment, the results of which are transmitted back to the assistive assessment server106for analysis and retention.

In some embodiments, the teacher102A may also share the shared session invite code with one or more additional teachers of the one or more teacher(s)102A. The teacher102A may facilitate transmission of the shared session invite code from the teacher device104A to the teacher devices104A of the one or more additional teachers in a similar manner as for the student device104B. In some embodiments, the teacher102A may share the code with the one or more additional teachers, who may each then enter the code using an application on their own teacher device104A. These additional teacher devices104A may transmit a request to join the shared session to the assistive assessment server106. The request may include the shared session invite code. The assistive assessment server106may validate the teacher devices104A based on the shared session invite code and may transmit the shared session token associated with the shared session to the teacher devices104A. The teacher devices104A may use the shared session token to join the shared session. In this manner, any number of teachers can join a shared session with the student102B for a simultaneous, remote evaluation. The teachers102A may interact with their respective devices to complete their own assessments of the student102B, the results of which are transmitted back to the assistive assessment server106for analysis and retention.

FIG. 2depicts an illustrative data flow200between various components of an illustrative system architecture for an assistive assessment platform in accordance with one or more embodiments of the disclosure. The architecture of the assistive assessment platform may include elements, such as: (1) a Teacher Scoring Experience (TSE), also referred to as a teacher device104A, a web-based presentation for creating shared assessment sessions and recording student responses; (2) a Student Assessment Experience (SAE), also referred to as a student device104B, a web-based presentation for assessments and item delivery of teacher-created shared assessment sessions; and/or (3) a teacher assistive scoring platform (TASP) Communication Server, also known as an assistive assessment server106, a virtual machine or cloud-based hosting solution where information is passed between the shared sessions of TSE and SAE. The TASP Server may also facilitate communication between assessment data and any third-party Learning Management Systems or Learning Record Stores.

FIG. 2depicts an example data flow for creating a shared session for an assistive assessment platform. At exchange1, a teacher102A may launch a TSE application on their web-enabled device (e.g., teacher device104A) and authenticate their credentials with the TASP Server. At exchange2, upon authentication, the teacher102A may be presented with a variety of available assessments via the TSE. The assessments may be designed specifically for multi-device proctoring within the assistive assessment platform. At exchange3, the teacher102A may select an assessment from the list of available assessments. The teacher102A may enter student information such, as name, class, grade, student identifier, or the like or the teacher102A may select the student102B from a list of students who were previously administered an assessment using the assistive assessment platform

At exchange4, a request for creating a shared session is sent to the TASP Server. The request may include an indication of the selected assessment and any student information provided. At exchange5, the TASP server may receive the request and may create a shared session, known as an Assessment Lobby, where messages between TSE and SAE are transmitted. The assessment lobby may be associated with an Assessment Lobby Invite Code, used to join the lobby, and an Assessment Lobby Token used for secure communication between both TSE- and SAE-clients with the TASP Server. At exchange6, the TSE may receive and retain the shared session information (e.g., the Assessment Lobby Invite Code and Token). The shared session information allows the TSE to communicate with the Assessment Lobby hosted on the TASP Server.

At exchange7, the Assessment Lobby Invite Code is displayed on the TSE for the teacher102A to share with the designated student102B. At exchange8, the student102B may launch the SAE on their web-enabled device (e.g., student device104B). At exchange9, the student102B may enter the Lobby Invite Code into the SAE. At exchange10, the SAE may generate and send a request to the TASP Server to join the Assessment Lobby identified by the invite code. At exchange11, the TASP Server may validate the SAE using the assessment lobby token. Upon validation, the TASP Server may send to the SAE the associated Assessment Lobby Token. At exchange12, the SAE and TSE may be synchronized to the same shared session hosted on the TASP Server and the assessment may be administered and scored accordingly.

In some embodiments, at exchange7the Assessment Lobby Invite Code displayed on the TSE for the teacher102A may also be shared by the teacher102A with other authenticated teachers102A who wish to join the Assessment Lobby and observe the assessment being administered. These additional teachers102A can launch their own TSEs on their own web-enabled devices (e.g., teacher devices104A) and can authenticate their credentials with the TASP Server in a similar manner as the initial teacher102A. These additional teachers102A may each enter the Lobby Invite Code into their TSE to request to the TASP Server to join the Assessment Lobby identified by the invite code in a similar manner as the student102B. The TASP Server may validate each of the TSEs using the assessment lobby token. Upon validation, the TASP Server may send to each validated TSE the associated Assessment Lobby Token. In this manner, the SAE and each of the validated TSEs may be synchronized to the same shared session hosted on the TASP Server and one or more individual teacher assessments may be administered and scored accordingly.

Allowing two or more teachers to provide simultaneous, individual teacher assessments of the same student offers several advantages over the use of a single evaluator. For example, the use of multiple evaluators allows for a comparison to be made between the evaluators themselves. Due to individual differences among teachers, two teachers may score the same response by the student102B differently. In some embodiments, the TASP Server tracks these differences, to determine, for example, the intra-reliability of the teacher assessments. This data can in turn be used to monitor for, and address, any discovered biases or inconsistencies in the teacher assessments.

In some embodiments, the TASP Server generates a score (also referred to as a reliability score) for each teacher. This score can be based at least in part on a question-by-question or overall comparison of the teacher's assessments to the average or most common assessments from all of the teachers. Alternatively, the assessment received from the initial teacher102A that set up the shared session can be considered the correct assessment, and deviations from this assessment in the other teacher's assessments can be tracked. In yet other embodiments, the assessment received from the teacher having the highest current score (from, e.g., previous assessments) can be considered the correct assessment, and deviations from this assessment in the other teacher's assessments can be tracked.

For example, if five teachers are evaluating a student, and only one of the five teachers scores question four as “incorrect,” the TASP Server can record and later report on this inconsistency. Similarly, the TASP Server can record and later report on one of the five teachers providing an assessment (for example, 80% correct) which differs significantly from the assessments provided by the other teachers (for example, an average of 45% correct). Over time, the TASP Server can determine which teachers are consistent outliers (e.g., those having low reliability scores) and which teachers provide evaluations without these inconsistencies (e.g., those having high reliability scores). In some embodiments, the TASP Server can use these scores to provide relative weights to the teacher assessments. For example, an assessment from a teacher having a low reliability score can be given less weight. Conversely, an assessment from a teacher having a high reliability score can be given more weight.

These reliability scores may or may not be visible to the teachers102A. In some embodiments, these scores are periodically provided to a teacher evaluator or to the teachers102A themselves for training purposes. In some embodiments, these reports can be triggered based at least in part on predetermined conditions. For example, a TASP Server report can indicate that a particular teacher has a sufficiently low reliability score (below, for example, a predetermined reliability threshold) to trigger an intercession.

FIG. 3is an illustrative diagram depicting in example educational assessment. The assistive assessment platform creates a shared assessment session for the assessment of oral or silent accuracy in educational assessment that both a student102B and a teacher102A can use simultaneously. As assessments move to be technology-based, groups are struggling with how to integrate student assessment and teacher scoring. The assistive assessment platform bridges the gap between student performance and teacher evaluation by synchronizing the experiences for both.FIG. 3depicts a sample letter name fluency task300where students102B are given 60 seconds to accurately state the name of the letter. The goal of the assistive assessment platform is to remove the to: 1) print forms for the letter name fluency task, 2) have the student102B and teacher102A use separate forms for performance and scoring, 3) have the teacher102A score the assessment, and 4) have the teacher102A manually enter a set of scores from this protocol. The assistive assessment platform transforms the protocol inFIG. 3to what is depicted inFIG. 4.

FIG. 4depicts illustrative embodiments of a student assessment experience400and a teacher scoring experience450in accordance with one or more embodiments of the disclosure. The student assessment experience400shows the presentation of the items as seen by the student102B, which may include an identification of the type of task for the assessment, an indication of the number of elements in the task, and the presentation of an element for the assessment. The teacher scoring experience450shows the presentation and scoring of items as seen by the teacher102A using a joint session. The teacher scoring device450will have control of the assessment time (e.g., Response Time) and will have the ability to stop the assessment after the pre-defined amount of time has been reached. The teacher scoring experience may show an identification of the student being assessed, an identification of the type of task for the assessment, an indication of the number of elements in the task, a depiction of the element for the assessment as shown to the student, and user interface elements that permit the teacher to score the task based on the response by the student102B.

In some embodiments, the user interface elements that permit the teacher to score the task include a “correct” answer element and an “incorrect” answer element. Alternatively or in addition to these answer elements, the user interface elements that permit the teacher to score the task can include one or more “allowable” responses. For example, if a multiple-choice question is posed to the student102B, the allowable responses can include each of the available answer choices, one or more of which can be selected by the teacher102A depending on the student response. The teacher102A can select between these answer elements depending on the student's response to the element presented for the assessment.

FIG. 5is an example process500flow diagram for an assistive assessment platform in accordance with one or more embodiments of the disclosure. At block505, a teacher device104A may be authenticated. In some embodiments, a teacher102A may interact with a teacher device104A using a web browser or a dedicated application. The teacher102A may provide authentication credentials (e.g., username and password, authentication token, etc.). The authentication credentials may be transmitted to the assistive assessment server106for authentication. The assistive assessment server106may authenticate the teacher102A based on the authentication credentials provided via the teacher device104A.

At block510, presentation of available assessments may be facilitated. In some embodiments, the authentication credentials may be associated with a profile associated with the teacher102A and/or teacher device104A. The assistive assessment server106may retrieve a listing of assessments available for the teacher102A and/or teacher device104A. In some embodiments, the assessments may be presented by grade level, assessment type, availability (e.g., based on type of teacher device104A or student device104B, assessments specific to an identified student102B, or the like). In some embodiments, the assessments may be designed specifically for multi-device proctoring, such as described in the context of the systems and methods described herein.

At block515, a request for creating a shared session may be received. The teacher102A may select from the list of available of assessments using the teacher device104A. In some embodiments, the teacher102A may provide student information via the browser or dedicated application associated with the assistive assessment platform. The teacher102A may enter student information, such as name, class identifier, grade level identifier, student identifier, or the like. In some embodiments, the teacher102A may select a student102B from a list indicative of students that previously used the assistive assessment platform. The web browser or dedicated application may generate and transmit a request for a shared session for an assessment to an assistive assessment server106.

At block520, a shared session may be generated. In some embodiments, the assistive assessment server106may generate the shared session (also known as the assessment lobby) based at least in part on the identified assessment to be administered, the student identified for the assessment, and the like. The assistive assessment server106may create the shared session where messages between the teacher device104A and the student device104B are transmitted. The shared session may be associated with shared session information. Shared session information may include, but is not limited to, the shared session invite code (also known as the assessment lobby invite code) and the shared session token (also known as the assessment lobby token). The shared session invite code may be a string that includes alphanumeric characters, symbols, and the like. In some embodiments, the transmission of the shared session invite code may be encrypted. The shared session token may be a token that is used to secure communication between the teacher device104A, student device104B, and the assistive assessment server106.

At block525, the share session information may be transmitted to the teacher device104A. The assistive assessment server106may generate a response and transmit the shared session information to the teacher device104A in response to the request for the shared session. In some embodiments, the teacher device104A may receive and retain the shared session information, which allows the teacher device104A to communicate with and through the shared session hosted by the assistive assessment server106.

In some embodiments, the shared session invite code may be displayed on the teacher device104A for the teacher102A to share with the designated student102B. The shared session invite code may be shared with the student102B by showing the student the shared session invite code on the teacher device104A (e.g., the student writes down the invite code) or by sharing the shared session invite code with the student (e.g., electronically via text, email, voice message, or the like). The shared session invite code may also be shared with other teachers associated with the assistive assessment platform (e.g., have the necessary authentication credentials and satisfy any pertinent or relevant rules to access to assessment associated or designated for the student). The other teachers may wish to join the shared session to observe the assessment administered to the designated student102B.

At block530, a request to join a session may be received from a student device104B. In some embodiments, the student102B may interact with a student device104B to launch a web browser or dedicated client application. The student device104B may transmit a request to join the shared session generated at the request of the teacher102A. The request may include the shared session invite code.

At block535, the student device104B may be validated. The assistive assessment server106may receive the request from the student device104B to join the shared session. The assistive assessment server106may validate the student device104B using the shared session invite code.

At block540, session information may be transmitted to the student device104B. The assistive assessment server106may transmit the shared session token associated with the shared session generated at the request of the teacher102A to the student device104B. The student device104B may receive the shared session token, which may be used to enable the student device104B to access the shared session. The student device104B and the teacher device104A may be synchronized to the same shared session hosted by the assistive assessment server106.

In some embodiments, the assistive assessment server106may act as the authority to all clients (e.g., student devices104B, teacher devices104A, etc.). For an identified assessment, elements of the assessment may be delivered or transmitted from the assistive assessment server106to the teacher device104A and the student device104B using formats associated with the respective type of client (e.g., student or teacher), as discussed with regard toFIG. 4. In some embodiments, the student102B may be presented with an element prompt via the student device104B and may respond verbally. The teacher102A may be presented with scoring information and user interface elements for recording the verbal response of the student102B via the teacher device104A. When the teacher102A has completed scoring the element using the teacher device104A, the scoring data may be transmitted to the shared session of the assistive assessment server106. The assistive assessment server106may retain the scoring data and assessment and deliver the next element of the assessment to the clients (e.g., student device104B, teacher device104A) or terminate the assessment based on predetermined stop criteria (e.g., timing, number of elements, obtaining a minimum threshold score, etc.) or a command received from either of the client devices.

Upon completion or termination of an assessment, the teacher device104A and student device104B may display their respective results. The teacher device104A may display detailed results to the teacher102A who can then confirm the data being recorded. The teacher102A may print or email the results, or if the assessment is integrated with a Learning Management Systems or Learning Record Store, can push the results to those endpoints. The student device104B, when appropriate, may also show the student their proficiency metrics based on the administered assessment. When the original teacher102A, who requested the shared session, exists the shared session, the assistive assessment server106may terminate the shared session and may set the shared session invite code as well as associated shared session tokens as expired, preventing any subsequent use of the shared session information.

FIG. 6is another example process flow diagram600for an assistive assessment platform, in accordance with one or more embodiments of the disclosure. The flow diagram600may illustrate an example of how automatic, real-time adjustments may be performed to an assessment provided to a student device. The flow diagram600may begin with operation602, which may involve presenting a first assessment stimulus on the student device. The assessment stimulus may be presented to the student device in any manner described herein. The flow diagram600may then proceed to operation604, which may involve a proctor scoring a response provided to the student device for the assessment stimulus. For example, a teacher may indicate by providing an input to the proctor device that a response provided to the student device was a correct or incorrect response to the assessment stimulus.

Following operation604, the flow diagram600may proceed to operation606. In some embodiments, operation606may involve determining a probability (Pi), which may be a probability that the student would get the assessment stimulus correct. As depicted in the figure, the probability, Pi, may be based on particular parameters (for example, a, b, and c), which, in some cases, may all be numerical values. In some instances, the parameter “a” may represent item discrimination, which may refer to the ability of a particular assessment element to be used to identify a difference between a student that has high ability and low ability in a particular area of interest that is being measured. The “b” parameter may represent item difficulty, which may refer to a difficulty of an assessment element that is provided to the student device (for example, the assessment element provided in operation702). In some cases, the “b” parameter may be an numerical value falling within a range (for example, −3 to 3, or any other numerical range). The “c” parameter may represent a pseudo guessing parameter, which may refer to a probability that the student guessed in providing an answer to an assessment element. For example, if the assessment stimulus is a multiple choice question with four possible answer choices, then the student may have a 25% chance of getting the question correct just by guessing. This c parameter accounts for this. In some cases, these parameters may be calibrated through an initial process, and may be added to a reference table that may be accessed by a system performing real-time updated of an assessment. That is, the reference table may include parameters associated with an assessment stimulus (“i”) that has just been administered to the student device. In this manner, each assessment stimulus may be associated with its own a, b, and c values, which may be stored in the reference table.

From operation606, the flow diagram600may iterate between presenting additional assessment stimulus and receiving proctor scorings for a given initial set of assessment stimulus provided to the student device. That is, the assessment may involve a set of pre-selected initial questions (FIG. 6provides one non-limiting example assessment that includes three additional initial assessment stimulus presented to the student device through operations620,624, and628respectively). After each of the initial assessment stimulus is presented and scored, the flow diagram600may iterate back through operation606to determine a Pifor that particular assessment stimulus. For example,FIG. 6may include four iterations including determining four different P, values for the four assessment stimulus that are initially provided to the student device. The initial set of assessment stimulus provided to the student device as the flow chart600iterates through the operation606may be pre-selected questions that may not involve real-time adjustments as may be performed in later operations in the figure (for example, operations608-618as described below). In other words, the initial set of questions may be presented and scored before the adaptive portion of the assessment begins. Additionally, although the figure illustrates the initial pre-selected assessment stimulus as including a set of four initial assessment stimulus, any other number of initial assessment stimulus may also be presented before the flow diagram600proceeds to operations608-618. In some cases, the set of pre-selected assessment stimulus may even include only one initial assessment stimulus with only one P, calculation being performed before operations608-618are performed. In even further case, the assessment may not include any fixed assessment stimuli, and all assessment stimuli may be determined adaptively and/or in real-time.

Following operation606(after all of the initial assessment stimulus have been presented and scored), the flow diagram600may proceed to operations608-618. These operations may represent the adaptive portion of the assessment, where real-time adjustments may be made to the assessment in terms of assessment stimulus that are selected to be presented to the student device after the initial assessment stimulus. For example, the assessment may be adaptive in that assessment stimulus being provided to the student device may be selected in real-time based on responses provided by the student to the student device to with respect to assessment stimulus already presented on the student device. The automatic real-time adjustments may also be used to determine whether the assessment should continue to present subsequent assessment stimulus on the student device, or whether sufficient information has been collected based on the assessment stimulus that have already been presented to the student device. For example, the information may relate to a student ability metric, which may be a value that is intended to be measured through the administration of the assessment to the student. That is, the student ability metric may be a numerical value representing an ability level of the student as determined through the responses being provided by the student to the assessment stimulus. If sufficient information has been collected, then the assessment may alternatively be ended without presenting any additional assessment stimulus, and the student ability metric and any responses provided to the student device may be stored. Additionally, while the figure illustrates operations608-618as being iterated through four rounds of assessment stimulus being provided to the student device, the operations608-618may be iterated any other number of times as well (including less than or greater than the four rounds of assessment stimulus). In some cases, only one iteration may be performed if it is determined that one round of assessment stimulus is sufficient to produce information about a student's ability that is intended to be measured by an assessment being presented to the student device.

In some embodiments, operation608may involve determining a set of likelihood values (L) of theta (θ) values, where the theta value may represent a student ability metric, and the likelihood values may represent a likelihood that a theta value is correct (for example, an accurate measurement of a student's ability that is being measured through a given assessment). These likelihood values may be determined for some or all of the probabilities (Pi) determined in operation616for the initial assessment stimulus provided to the student device. For example, the student's ability represented by the theta value may be a metric that is intended to be obtained through the presentation of the assessment to the student through the student device. More specifically, operation608may involve computing the probability that the assessment stimulus would be answered correctly for different increments of theta values (for example, a theta value of −3, a theta value of −2.9, −2.8, etc., as well as any other increments of theta). That is, the probability, P, determined in operation606may be determined for a range of values to produce a resulting group of likelihood values (which may form a plot of likelihood values in the form of a bell curve or any other type of curve). Based on the resulting values, a maximum likelihood may be identified (for example, a maximum point on the bell curve (if the curve is a bell curve)). This maximum likelihood may represent the most likely theta value for the student based on the answers provided to the student device with respect to the assessment stimulus that have already been presented on the student device during the assessment. Once the maximum likelihood value is determined, the associated theta value may be used as an interim theta. This interim theta value may represent an interim student ability value that may be refined as additional iterations of the operations are performed (additional assessment stimulus are identified and provided). In other words, operations606and608may essentially involve the creation in real-time of a look-up table in order to calculate a theta value based on responses. For each item (or only some items) administered, the probability of correct response P is calculated across the theta range (−3 to 3, or any other range). The P values may then be summed across items, and the ability may be estimated at the theta value where the sum is maximized. Additionally, this may only be one example of a method by which a theta value may be estimated, and other methods may be used as well.

Following operation608, the flow diagram600may proceed to operation610. Operation610may involve determining “information.” In some cases, the information may be Fisher Information, which may be a measure of how much information about an unknown parameter may be obtained from a data sample (for example, how well a parameter may be measured, given a certain amount of data). The information formula may take into account an item's discrimination, difficulty, and pseudo-guessing parameters at a specific ability or theta value.

Following operation610, the flow diagram600may proceed to operation612. Operation612may involve determining a standard error (SE) measurement. The SE measurement may represent a confidence value serving as a measure of the accuracy of the current interim theta value (for example, student ability). The SE measurement may be used to determine if the assessment should continue to adaptively provide additional assessment stimulus to the student device or if the interim student ability determined based on the already-presented assessment stimulus is sufficient accurate for the assessment to be completed. For example, at condition614, it may be determined if the SE measurement is within a given threshold value. In the figure, the condition614provides the example of the SE being less than a numerical value of 0.316, however, any other value may also be used. If it is determined in condition614that the SE measurement satisfies the threshold determination, then the assessment may end and the student ability as well as the responses provided by the student may be stored. That is, it may be determined that the assessment stimulus provided to the student device, and the responses received from the student for those assessment stimulus, are sufficient to determine the student's ability. Otherwise, if it is determined in condition614that the SE measurement does not satisfy the threshold determination, then additional assessment stimulus may be adaptively provided to the student device in the manner described herein until the SE measurement falls within the particular threshold value. In this case, the flow diagram600may proceed to operation616.

Following operation614(if it is determined in condition614that the SE measurement does not satisfy the threshold determination), the flow diagram600may proceed to operation616. Operation616may involve determining a type of assessment stimulus to provide to the student device if the SE measurement does not satisfy the threshold and additional assessment stimulus need to be provided to the student device. In some cases, the assessment stimulus may not necessarily be a randomly-generated assessment stimulus or a fixed assessment stimulus, but may rather be an assessment stimulus that is tailored to the particular student based on the interim theta value calculated based on the previous answers provided by to the student device, where the theta value may be the interim theta value determined in operation610. The operation616may involve identifying an assessment stimulus that is closest to the current theta value for the student. For example, if the theta value is currently1for a student, then a pool of available assessment stimulus may be referenced to identify one or more assessment stimulus that may have an item difficulty (for example, the “b” parameter associated with the assessment stimulus) closest to a theta value of 1. For example, such an assessment stimulus may be an assessment stimulus that a student with a theta of 1 should be able to answer correctly (or has a given percentage chance of answering correctly). This assessment stimulus (or group of assessment stimulus may then be presented to the student device (for example, at operation620). This may only be one non-limiting example of a method by which the additional assessment stimulus is determined. Another example method may include selecting the assessment stimulus with the most information for the current ability estimate of the student (for example, using the information look-up table).

The above operations may then be iterated any number of times until it is determined that the SE value is below the threshold (for example, when it is determined that the confidence value indicates that the theta value (student ability) is within a given confidence range of accurately representing the ability of the student. The final theta value may be the metric that may represent the desired output of the assessment provided to the student.

It should be noted that althoughFIG. 6presents one example set of operations for automatically updating an assessment in real-time, the real-time updates may be performed using any other operations as well. Additionally, the operations may be implemented using any type of artificial intelligence, deep learning, and the like. Furthermore, as more assessment stimulus are presented to different student devices, the artificial intelligence, deep learning, and the like may be further trained to be able to better identify subsequent assessment stimulus to adaptively provide to the student device.

FIG. 7schematically illustrates an example architecture of an assessment piloting platform701, in accordance with one or more embodiments of the disclosure. In some embodiments, the assessment piloting platform701may be a collection of web-based and native iOS, Android, macOS, and Windows applications for creating and piloting items for assessment delivery. The assessment piloting platform701may include a portal that may be a web-based system where users can manage students, create assessments, enroll students in assessments, view and export collected scores, and manage test administrators. The portal may interface with assessment delivery applications through API services. The assessment delivery applications (for example, a proctor application728and/or a student application729) may utilize the teacher assistive scoring platform (TASP) technology described herein for administering items that require an oral response from the examinee and a manual scoring by a test proctor. The assessment piloting platform system may be responsible for the authorship, delivery, and reporting of piloted assessments. This includes functionality to manage users, students, and enrollments. A description of the components of this platform may be as follows.

In some embodiments, the assessment piloting platform701may include one or more associated systems. For example the assessment piloting platform701may include an assessment management system704, a student management system705, an enrollment management system706, a user management system707, a computerized adaptive test (CAT) data management system708, and/or a reporting system709. The assessment management system704may include assessment data710, task data711, item banks data712, and/or items713. In some cases, the items713may include at least audio media714and/or visual media715. The assessment data710may access CAT configuration data716, which may include parameters717, models718, administrative criteria719, and/or reporting criteria720. The CAT data management interface708may include item parameters data721, lookup tables722, and/or reporting criteria723. In some cases, item parameters data721may include the discrimination, difficulty, and pseudo-guessing parameters (a,b,c) for each item. Lookup tables722and reporting criteria723may refer to predetermined developmental scale scores and percentile ranks based on a student's estimated ability score. The student management system705may allow for the creation of student tracking IDs. This student management system705may also support bulk import and export, as well as marking which students are currently active in the system. Assessment scores for inactive students may be retained, but may not be included in scoring reports, in some cases. An enrollment may be defined as the specific association of an assessment to a student for a particular year, assessment period (AP), district, school, and teacher. the enrollment management system706may allow users to bulk import and export enrollments (for example, through the use of Excel documents or through any other suitable methods). The user management system707may allow administrators to control access for proctors to the system. A user may be defined by an email address, first and last name, role, and district. The district a user is assigned to may limit their access to enrollments only from that district. Testers may be required to sign in and authenticate within the proctor application in order to access enrollments and administer assessments. The assessment piloting platform701may support the piloting of computerized adaptive tests (CAT). The CAT data management system708may allow for the configuration of CAT tasks and assessments. The CAT data management interface708may include item parameters data721, lookup tables722, and/or reporting criteria723. The reporting system709of the assessment piloting platform701may allow for student scores and audio recordings to be exported. The reporting interface709may include a scores export724, an audio export725, and/or a CAT reporting export726. Users may filter by date ranges.

In some embodiments, an internal API703may allow for communications between the assessment piloting platform701to one or more assessment delivery applications, such as a proctor application728and/or a student application729. This may include at least authentication of users, fetching assessment data and assets, fetching student enrollments, post student scores, and/or posting audio recordings of student responses. The proctor application728may be an application that may be used by a tester to connect to a student device and administer an assessment to the student. The student application729may be used by a student through a student device to take an assessment provided to the student device by the proctor application728of the proctor device. These applications may fetch data from the internal API703. Additionally, the proctor application728may post student scores while the student application729may post audio recordings of student responses. Video chat functionality may also integrated into the applications in order to facilitate remote administration of assessments from the proctor application728to the student application729. The internal API703may also be responsible for the publishing of assessments to a dissemination middleware system730. After conducting research and analysis on item and assessment performance, the dissemination middleware system730may allow authors to “publish” assessment assets for public distribution. The dissemination middleware system730may include two primary components: (1) an assessment assets and data repository731, and (2) an external API743. The assessment assets and data repository731may be a system where items and assessments authored in the internal assessment piloting platform701may be stored for public distribution. This may include the specific media assets, such as audio or graphics, as well as the relationship between items, item banks, tasks, and assessments. Included also may be the specific computer adaptive testing (CAT) configurations needed for runtime delivery. In some cases the data may include assessment data710, task data711, item banks data712, and/or items713. In some cases, the items713may include at least audio media714and/or visual media715. The assessments710may access CAT configuration data716, which may include parameters717, models718, administrative criteria719, and/or reporting criteria720. This data may be the same as the data connected to the assessment management interface704described above. The data may also be different as well.

In some embodiments, when an administrator publishes or updates an assessment, the assessment piloting platform701may create a static snapshot of the assessment data structure, along with the associated tasks, forms, item banks, items, and media assets. The metadata for the snapshot may then be stored in a separate table in the database, and the media files may be moved to a public-facing staging environment specific to the dissemination middleware730. The URLs pointing to the media assets may be modified to specify their new location in the staging environment. Assessment authorship and piloting may occur within a closed system. Once validation and research has concluded, finalized assessment forms may be published to the dissemination middleware730for use by external parties (for example, third party applications744. The assessment piloting platform701may work in conjunction with the dissemination middleware730to promote assessment assets and media for use with the third party applications744. The dissemination middleware730may be connected to third party applications744through the external API743, where the third party applications744may include assessment delivery systems745and/or enrollment systems746. The dissemination middleware730may also be connected to certain types of data though an assessment asset and data repository API.

FIG. 8schematically illustrates an example structure of assessment data, in accordance with one or more embodiments of the disclosure. The structure of assessment data may be defined by a relationships between an assessment (for example, assessment802), tasks (for example, task804), forms (for example, form806, form808, form810, and/or form812), items banks (for example, item bank814, item bank816, and/or item bank818), and items (for example, item820, item822, and/or item824, as well as any other items depicted in the figure). An assessment may be a collection of tasks, which may be defined by their type. A task may refer to types of items that are presented to a student during an assessment. For example, a task type may include a vocabulary task, listening comprehension, spelling, etc. A task may allow for multiple forms. A form may be an instance of a specific item bank associated with a specific task. Continuing the above example with the task being a vocabulary task, there may be two different item banks (a first item bank and a second item bank). It may be desired for a third of students to get the first item bank and two thirds of the students to get the second item bank. To this end, three “forms” (for example, a first form, second form, and third form) may be generated, where the first form may use the first item bank, and the second and third forms may use the second item bank. The vocabulary task may randomly select one of these forms (for example, the first form, second from, and third firm) to present. This may ensure the goal of the first item bank being used approximately a third of the time and the second item bank being used two thirds of the time. An item bank may be a collection of items, which may be defined by: item name, prompt text, prompt image, prompt audio, options, etc. An item may also be referred to herein as an “assessment stimulus,” or may comprise an assessment stimulus (that is, an assessment stimulus may refer to a “question” being presented to a student). An option within an item may be defined by option name text, image, audio, is correct, etc. That is, an option may refer to potential responses that may be provided by a student. This data structure and the relationships between models may allow for flexibility in the piloting and authorship of assessments. As one specific example, an item may include an assessment stimulus in the form of a picture of an animal and a question asking what the animal is called. The item may also include one or more options in the form of multiple choice answers for the student to select from.

In some embodiments, tasks may have different template types that dictate how the tasks are presented and interfaced with. Tasks may include forms of item banks, which may be collections of items. Examples of tasks currently may include: proctor directions, audio only, image audio reveal, option image student select, digit span, sentence repetition, timed naming, kerning grid, reading passage, spelling, complete. In some cases, proctor directions may involve directions the proctor reads to the student. Audio only may involve the student being presented only audio stimuli and being scored by the proctor. Image audio reveal may involve the student being presented both audio and visual stimuli and being scored by the proctor. Option image student select may involve the student being presented visual stimulus and selecting a response on screen. Digit span may involve the student being presented only audio stimuli of numbers to recite and being scored by the proctor through a number pad interface. Sentence repetition may involve the student being presented only audio stimuli of a sentence to repeat and being scored at the individual word level by the proctor. Timed naming may involve the student being presented a grid of letters or images, being required name them as quickly as possible, and being scored by the proctor. Kerning grid may involve a grid of images being presented and the student must find a specific image or glyph in the grid, which the student may do by selecting the image or glyph on screen. Reading passage may involve a passage being presented to the student to read and the proctor scoring any errors or missing words. Spelling may involve tiles with letters being presented on screen and the student must assemble them in the correct order based on an audio prompt. These are merely examples of potential tasks, and are not intended to be limiting.

FIGS. 9A-9Jdepicts an example user interface for an admin portal of the assessment piloting platforming, in accordance with one or more embodiments of the disclosure. For example, the admin portal may allow a system administrator to log into the assessment piloting platform (for example, the assessment piloting platform701) to perform tasks such as registering and assigning students to assessments, managing proctor access, exporting scores and captured audio files, uploading CAT configurations. It should be noted that the user interface illustrated inFIGS. 9A-9J, as well asFIGS. 10A-10DandFIG. 11are merely exemplary, and not intended to be limiting or to be the only user interface that may be presented to a user. Any other number and/or types of user interface may also be presented as well. Additionally, elements included within the user interface depicted in any of the figures may be modified in any suitable manner and may not necessarily be limited to the exact configuration illustrated in the figures.FIG. 9Amay present a first user interface900of the admin portal. The first user interface900may include one or more selectable elements that may allow the administrator to access different systems of the assessment piloting platform. For example, a first selectable element901may allow the administrator to access a student management system, a second selectable element902may allow the administrator to access an assessment management system, a third selectable element903may allow the administrator to access an enrollment management system, a fourth selectable element904may allow the administrator to access a user management system, and/or a fifth selectable element905may allow the administrator to access an reporting system. In some cases, assessment management system, student management system, enrollment management system, user management system, and/or a reporting system may be the same as the assessment management system704, student management system705, enrollment management system706, user management system707, and/or reporting system709mentioned above. The user interface900may also present any other number of selectable elements as well.

FIGS. 9B and 9Cdepict a second user interface906and a third user interface907of the admin portal. The second user interface906may illustrate an example student listing that may be presented to the administrator. The second user interface906may also allow the administrator to edit student information, export student information, bulk import student information, or add individual students. The third user interface907may illustrate an example assessment listing. The third user interface907may allow the administrator to edit assessments, create new assessments, and access tasks, item banks, and/or individual items.

FIGS. 9D-9Gdepict a fourth user interface908, a fifth user interface909, a sixth user interface909, and a seventh user interface910of the admin portal. The fourth user interface908may allow the administrator to create and/or edit an individual assessment. The fifth user interface909may allow the administrator to create and/or edit individual tasks in a task list. The sixth user interface910may specifically illustrate how the administrator may create a new task. The seventh user interface911may allow the administrator to create and/or edit individual items in an item bank list.

FIGS. 9H-9Jdepict an eighth user interface912, a ninth user interface913, and a tenth user interface913of the admin portal. The eighth user interface912may allow the administrator to create and/or edit an item bank. The ninth user interface913may allow the administrator to create and/or edit individual items, including adding text, image, and/or audio options, and an indication of a correct answer that a student may select on a student device during an assessment including the particular item. The tenth user interface914may allow the administrator to perform reporting operations, such as exporting student scores and/or exporting audio recordings of student responses.

FIGS. 10A-10Ddepicts an example user interface for a proctor portal of the assessment piloting platforming, in accordance with one or more embodiments of the disclosure. The portal may allow a proctor (for example, a teacher or any other person who is administering a test to a student) to administer an assessment to one or more student devices. For example, the proctor portal may be the same as the proctor application728mentioned above.FIG. 10Amay present a first user interface1000of the admin portal,FIG. 10Bmay present a second user interface1001of the proctor portal.FIG. 10Cmay present a third user interface1002of the proctor portal.FIG. 10Dmay present a fourth user interface1003of the proctor portal. The first user interface1000may present information to the proctor about the status of various students. For example, the user interface1000may present a listing of students that have completed an assessment, a listing of students that are currently taking an assessment, and a listing of students that have not taken an assessment. The student listing can be filtered based on school, teacher, or any other criteria. The first user interface (as well as the second user interface1001, third user interface1002, and fourth user interface1003) may also present an invitation code that may be provided to a student device to allow a proctor device to connect with the student device so that an assessment may be provided to the student device. The second user interface10001may allow a proctor to begin an assessment with a particular student. The second user interface1001may also present information about the particular student. The third user interface1003may allow the proctor to pause the assessment, restart the assessment, and/or resume the assessment. The fourth user interface1004may provide an indication to the proctor when an assessment is completed at a student device.

FIG. 11depicts an example user interface for a task portal of the assessment piloting platforming, in accordance with one or more embodiments of the disclosure. For example, withinFIG. 11, the interface1100may be associated with a proctor device, and the interface1102may be associated with a student device. The interface110may include a task that a proctor may provide to the interface1102of the student device during an assessment. For example, the figure may illustrate a particular task being provided to the proctor device at the interface1100. The interface1100may present selectable elements that may allow the proctor to present portions of the task to the interface1102for presentation on the student device. For example, a first selectable element1104associated with a prompt of the task may be selected on the interface1100, which may correspondingly present the prompt1112on the interface1102of the student device. The selectable element1106, selectable element1108, and/or selectable element1110may also be selected on the interface1100, which may present options associated with the prompt on the interface1102of the student device as the first option1114, second option1116, and/or third option1118. These options may represent potential responses to the prompt1112. It should be noted that this is merely one non-limiting example of how an assessment may be presented to a student device. The assessment stimuli may be presented in any other manner as well. For example, the prompt1112and/or options (1114,116, and1118) may be automatically presented to the interface1102all at once, or at different intervals of time. In this sense, the administration of the assessment stimuli themselves may not necessarily require any input from the proctor beyond scoring the responses provided by the student.

Illustrative Computer Architecture

FIG. 12is a schematic block diagram of one or more illustrative assistive assessment server(s)1200in accordance with one or more example embodiments of the disclosure. The assistive assessment server(s)1200may include any suitable computing device including, but not limited to, a server system, a mobile device such as a smartphone, a tablet, an e-reader, a wearable device, or the like; a desktop computer; a laptop computer; a content streaming device; a set-top box; or the like. The assistive assessment server(s)1200may correspond to an illustrative device configuration for the assistive assessment servers ofFIGS. 1-5.

The assistive assessment server(s)1200may be configured to communicate via one or more networks with one or more servers, user devices, or the like. The assistive assessment server(s)1200may be configured to coordinate transmission of data between teacher devices104A and student devices104B to administer assessments and record the scoring data submitted by the teacher device104A based on administration of the assessment via the teacher device104A and student device104B.

In an illustrative configuration, the assistive assessment server(s)1200may include one or more processors (processor(s))1202, one or more memory devices1204(generically referred to herein as memory1204), one or more input/output (I/O) interface1206, one or more network interface1208, one or more sensors or sensor interface1210, one or more transceivers1212, one or more optional speakers1214, one or more optional microphones1216, and data storage1220. The assistive assessment server(s)1200may further include one or more buses1218that functionally couple various components of the assistive assessment server(s)1200. The assistive assessment server(s)1200may further include one or more antenna(e)1234that may include, without limitation, a cellular antenna for transmitting or receiving signals to/from a cellular network infrastructure, an antenna for transmitting or receiving Wi-Fi signals to/from an access point (AP), a Global Navigation Satellite System (GNSS) antenna for receiving GNSS signals from a GNSS satellite, a Bluetooth antenna for transmitting or receiving Bluetooth signals, a Near Field Communication (NFC) antenna for transmitting or receiving NFC signals, and so forth. These various components will be described in more detail hereinafter.

The data storage1220may include removable storage and/or non-removable storage including, but not limited to, magnetic storage, optical disk storage, and/or tape storage. The data storage1220may provide non-volatile storage of computer-executable instructions and other data. The memory1204and the data storage1220, removable and/or non-removable, are examples of computer-readable storage media (CRSM) as that term is used herein.

The data storage1220may store computer-executable code, instructions, or the like that may be loadable into the memory1204and executable by the processor(s)1202to cause the processor(s)1202to perform or initiate various operations. The data storage1220may additionally store data that may be copied to the memory1204for use by the processor(s)1202during the execution of the computer-executable instructions. Moreover, output data generated as a result of execution of the computer-executable instructions by the processor(s)1202may be stored initially in the memory1204, and may ultimately be copied to the data storage1220for non-volatile storage.

More specifically, the data storage1220may store one or more operating systems (O/S)1222; one or more database management systems (DBMS)1224; and one or more program module(s), applications, engines, computer-executable code, scripts, or the like such as, for example, one or more data management module(s)1226and/or one or more assistive assessment module(s)1228. Some or all of these module(s) may be sub-module(s). Any of the components depicted as being stored in the data storage1220may include any combination of software, firmware, and/or hardware. The software and/or firmware may include computer-executable code, instructions, or the like that may be loaded into the memory1204for execution by one or more of the processor(s)1202. Any of the components depicted as being stored in the data storage1220may support functionality described in reference to corresponding components named earlier in this disclosure.

The data storage1220may further store various types of data utilized by components of the assistive assessment server(s)1200. Any data stored in the data storage1220may be loaded into the memory1204for use by the processor(s)1202in executing computer-executable code. In addition, any data depicted as being stored in the data storage1220may potentially be stored in one or more datastore(s) and may be accessed via the DBMS1224and loaded in the memory1204for use by the processor(s)1202in executing computer-executable code. The datastore(s) may include, but are not limited to, databases (e.g., relational, object-oriented, etc.), file systems, flat files, distributed datastores in which data is stored on more than one node of a computer network, peer-to-peer network datastores, or the like. InFIG. 12, an example datastore(s) may include, for example, web content, advertisement campaigns, advertisements, assessment information, content items, and/or other information.

Referring now to functionality supported by the various program module(s) depicted inFIG. 12, the data management module(s)1226may include computer-executable instructions, code, or the like that responsive to execution by one or more of the processor(s)1202may perform functions including, but not limited to, managing requests and responses received from teacher devices104A and student devices104B, identifying and obtaining assessments, transmitting data to the assistive assessment module1228, and the like.

The assistive assessment module(s)1228may include computer-executable instructions, code, or the like that responsive to execution by one or more of the processor(s)1202may perform functions including, but not limited to, generating shared session information, such as the shared session invite code and shared session token, administering assessments to an identified student102B by transmitting elements of the assessment to the teacher device104A and student device104B, receiving responses and scoring data from the respective devices, terminating the shared session, and the like.

Referring now to other illustrative components depicted as being stored in the data storage1220, the O/S1222may be loaded from the data storage1220into the memory1204and may provide an interface between other application software executing on the assistive assessment server(s)1200and the hardware resources of the assistive assessment server(s)1200. More specifically, the O/S1222may include a set of computer-executable instructions for managing hardware resources of the assistive assessment server(s)1200and for providing common services to other application programs (e.g., managing memory allocation among various application programs). In certain example embodiments, the O/S1222may control execution of the other program module(s) to dynamically enhance characters for content rendering. The O/S1222may include any operating system now known or which may be developed in the future including, but not limited to, any server operating system, any mainframe operating system, or any other proprietary or non-proprietary operating system.

The DBMS1224may be loaded into the memory1204and may support functionality for accessing, retrieving, storing, and/or manipulating data stored in the memory1204and/or data stored in the data storage1220. The DBMS1224may use any of a variety of database models (e.g., relational model, object model, etc.) and may support any of a variety of query languages. The DBMS1224may access data represented in one or more data schemas and stored in any suitable data repository including, but not limited to, databases (e.g., relational, object-oriented, etc.), file systems, flat files, distributed datastores in which data is stored on more than one node of a computer network, peer-to-peer network datastores, or the like. In those example embodiments in which the assistive assessment server(s)1200is a mobile device, the DBMS1224may be any suitable lightweight DBMS optimized for performance on a mobile device.

Referring now to other illustrative components of the assistive assessment server(s)1200, the input/output (I/O) interface(s)1206may facilitate the receipt of input information by the assistive assessment server(s)1200from one or more I/O devices as well as the output of information from the assistive assessment server(s)1200to the one or more I/O devices. The I/O devices may include any of a variety of components such as a display or display screen having a touch surface or touchscreen; an audio output device for producing sound, such as a speaker; an audio capture device, such as a microphone; an image and/or video capture device, such as a camera; a haptic unit; and so forth. Any of these components may be integrated into the assistive assessment server(s)1200or may be separate. The I/O devices may further include, for example, any number of peripheral devices such as data storage devices, printing devices, and so forth.

The assistive assessment server(s)1200may further include one or more network interface(s)1208via which the assistive assessment server(s)1200may communicate with any of a variety of other systems, platforms, networks, devices, and so forth. The network interface(s)1208may enable communication, for example, with one or more wireless routers, one or more host servers, one or more web servers, and the like via one or more networks.

The antenna(e)1234may include any suitable type of antenna depending, for example, on the communications protocols used to transmit or receive signals via the antenna(e)1234. Non-limiting examples of suitable antennae may include directional antennae, non-directional antennae, dipole antennae, folded dipole antennae, patch antennae, multiple-input multiple-output (MIMO) antennae, or the like. The antenna(e)1234may be communicatively coupled to one or more transceivers1212or radio components to which or from which signals may be transmitted or received.

The antenna(e)1234may additionally, or alternatively, include a Wi-Fi antenna configured to transmit or receive signals in accordance with established standards and protocols, such as the IEEE 802.11 family of standards, including via 2.4 GHz channels (e.g., 802.11b, 802.11g, 802.11n), 5 GHz channels (e.g., 802.11n, 802.11ac), or 60 GHz channels (e.g., 802.11ad). In alternative example embodiments, the antenna(e)1234may be configured to transmit or receive radio frequency signals within any suitable frequency range forming part of the unlicensed portion of the radio spectrum.

The transceiver(s)1212may include any suitable radio component(s) for—in cooperation with the antenna(e)1234—transmitting or receiving radio frequency (RF) signals in the bandwidth and/or channels corresponding to the communications protocols utilized by the assistive assessment server(s)1200to communicate with other devices. The transceiver(s)1212may include hardware, software, and/or firmware for modulating, transmitting, or receiving—potentially in cooperation with any of antenna(e)1234—communications signals according to any of the communications protocols discussed above including, but not limited to, one or more Wi-Fi and/or Wi-Fi direct protocols, as standardized by the IEEE 802.11 standards, one or more non-Wi-Fi protocols, or one or more cellular communications protocols or standards. The transceiver(s)1212may further include hardware, firmware, or software for receiving GNSS signals. The transceiver(s)1212may include any known receiver and baseband suitable for communicating via the communications protocols utilized by the assistive assessment server(s)1200. The transceiver(s)1212may further include a low noise amplifier (LNA), additional signal amplifiers, an analog-to-digital (A/D) converter, one or more buffers, a digital baseband, or the like.

The speaker(s)1214may be any device configured to generate audible sound. The microphone(s)1216may be any device configured to receive analog sound input or voice data.

One or more operations of the methods, process flows, and use cases may be performed by a device having the illustrative configuration depicted inFIG. 12, or more specifically, by one or more engines, program module(s), applications, or the like executable on such a device. It should be appreciated, however, that such operations may be implemented in connection with numerous other device configurations.