Package: openla-feature-representation
openla-feature-representation: generate features for EventStream dataIntroductionopenla-feature-representation is an open-source Python module that generates features fromOpenLAEventStream data, to make the data easier to use for ML.InstallationThis module module is available on PyPI and it can be installed usingpipas follows:pipinstallopenla-feature-representationDownloading the modelFor the E2Vec class, you will need theopenla-feature-representation-fastText_1min.binmodel .
Feel free to dowload it from theOpenLA models download site.Usage of the E2Vec classFirst, import theopenla_feature_representationpackage with an arbitrary name, herelafr.importopenla_feature_representationaslafrInitializing the classThis is the constructor:e2Vec=lafr.E2Vec(ftmodel_path,input_csv_dir_path,course_id)ftmodel_pathis the path to a fastText language model trained for this taskinput_csv_dir_pathis the path to a directory with the dataset (see below)course_idis a string to identify files for the course to analyze within theinfo_dirdirectory (e.g.'A-2023')After getting your owne2Vecobject, all methods the class provides can be used on it.Generate sentences for the event logThe fastText model uses an artificial language to express event log entries as sentences. This is how you can generate them:sentences=e2Vec.generate_sentences(sentences_dir_path=sentences_dir_path,eventstream_file_path=eventstream_file_path,input_csv_dir_path=input_csv_dir_path,course_id=course_id,)If you need to select or filter a time span:sentences=e2Vec.generate_sentences(sentences_dir_path=sentence_path,use_timespan=True,start_minute=0,total_minutes=90,eventstream_file_path=eventstream_file_path,input_csv_dir_path=input_csv_dir_path,course_id=course_id,)sentences_dir_pathis the path to the directory where you want the sentence files to be writteneventstream_file_pathis the path to the event stream csv fileinput_csv_dir_pathis the path to a directory with the dataset (see below)course_idis a string to identify files for the course to analyze within theinfo_dirdirectoryuse_timespanifTrue, the args below will be used to extract a timespan from the data (optional)start_minuteis the minute in the data the sentence generation should start (optional)total_minutesis the number of minutes worth of sentences that should be generated (optional)This function saves the sentences to a text file and returns a path to it.Vectorize the sentencesThis function returns a pandas DataFrame with the vectors generated from the sentences.user_vectors=e2Vec.vectorize_sentences(sentences_file_path)sentences_file_pathis the path to the sentence files generated in the previous stepConcatenationThe class has a function to concatenate vectors by time (minutes) or weeks.This will concatenate the vectors in 10-minute spans.vectors=e2Vec.concatenate_vectors(sentences_dir_path=sentences_dir_path,eventstream_file_path=eventstream_file_path,input_csv_dir_path=eduData,course_id=course_id,start_minute=0,total_minutes=10,)This will concatenate the vectors by the week or lesson.vectors=e2Vec.concatenate_vectors(sentences_dir_path=sentences_dir_path,eventstream_file_path=eventstream_file_path,input_csv_dir_path=eduData,course_id=course_id,by_weeks=True,start_minute=0,)sentences_dir_pathis the path to the sentence files generated in the previous stepeventstream_file_pathis the path to the event stream csv fileinput_csv_dir_pathis the path to a directory with the dataset (see below)course_idis a string to identify files for the course to analyze within theinfo_dirdirectoryby_weeksconcatenates vectors by week ifTrue(by time by default)start_minuteis the minute in the data the sentence generation should start (optional)total_minutesis the number of minutes worth of sentences that should be generated each time (optional)Usage of the ALP classALP (Active Learner Point) is a set of metrics that take BookRoll (ebook) and Moodle activity per lecture into account: attendance, report submissions, course views, slide views, adding markers or memos, and other actions.fromopenla_feature_representationimportAlpalp=Alp(course_id="114")course_idis a string to identify files for the course to analyze within theDatasetdirectoryTheAlpclass constructor above makes three DataFrames available as properties of the returnedAlpobject:features_df: aggregated totals of how many times each user took any of the relevant actions for each lecturealp_df: the features above replaced by a number from 0 to 5 following the criteria belowalp_df_normalized: same as above, only the 0 to 5 numbers are normalized between 0 and 1Criteria for the ALP 0-to-5 scalevaluedescription5:Top 10%, or attending the lecture, or submitting a report4:Top 20%3:Top 30%, or being late to the lecture, or submitting late2:Top 40%1:Top 50%0:Bottom 50%, or not attending, or not submittingExamples for the class's methods and instance variablesalp.features_df# These are the aggregated featuresalp.alp_df# These are the ALP 0-to-5 valuesalp.alp_df_normalized# These are the normalized values# The following will write CSV files for the relevant DataFrame# Paths and filenames can be specified, but there are default# filenames and the paths default to the present directoryalp.write_features_csv()alp.write_alp_csv()alp.write_alp_normalized_csv()Datasets for OpenLAThis module uses data in the same or a similar format as OpenLA. Please refer to theOpenLA documentationfor further information.Datasets for theAlpclass must be placed in the"Dataset/"directory relative to the Python script they are called from.