Package: hyperreal
HyperrealHyperreal is a Python tool for interactive qualitative analysis of large
collections of documents.RequirementsHyperreal requires the installation ofthe Python programming language.InstallationHyperreal can be installed using Pip from the command line (Windows,Mac)
by running the following commands:python -m pip install hyperrealUsageHyperreal can be used in three different ways to flexibly support different
use cases:as a command line applicationas a Python libraryas a local web applicationAll of hyperreal's functionality is available from the Python library, but you
will need to write Python code to use it directly. The command line interface
allows for quick and repeatable experimentation and automation for standard
data types - for example if you often work with Twitter data the command line
will allow you to rapidly work with many different Twitter data collections.
The web application is currently focused solely on creating and interactive
editing of models.Command LineThe following script gives a basic example of using the command line interface
for hyperreal. This will work for cases where you have a plain text file
(here calledcorpus.txt), with eachdocumentin the collection on its own
line.If you haven't worked with the command line before, you might find the
following resources useful:Software Carpentry resources for MacOpen Water Foundation resources for Windows# Create a corpus database from a plaintext file
hyperreal plaintext-corpus create corpus.txt corpus.db

# Create an index from the corpus
hyperreal plaintext-corpus index corpus.db corpus_index.db

# Create a model from that index, in this case with 128 clusters and
# only include features present in 10 or more documents.
hyperreal model corpus_index.db --min-docs 10 --clusters 128

# Use the web interface to serve the results of that modelling
# After running this command point your web browser to http://localhost:8080
hyperreal plaintext-corpus serve corpus.db corpus_index.dbLibraryThis example script performs the same steps as the command line example.fromhyperrealimportcorpus,index# create and populate the corpus with some documentsc=corpus.PlainTextSqliteCorpus('corpus.db')withopen('corpus.txt','r')asf:# This will drop any line that has no text (such as a paragraph break)docs=(lineforlineinfifline.strip())c.replace_docs(docs)# Index that corpus - note that we need to pass the corpus object for# initialisation.idx=index.Index('corpus_index.db',corpus=c)# This only needs to be done once, unless the corpus changes.idx.index()# Create a model on this index, with 128 clusters and only including features# that match at least 10 documents.idx.initialise_clusters(n_clusters=128,min_docs=10)# Refine the model for 10 iterations. Note that you can continue to refine# the model without initialising the clusters.idx.refine_clusters(iterations=10)# Inspect the output of the model using the index instance (currently quite# limited). This will print the top 10 most frequent features in each# cluster.forcluster_idinidx.cluster_ids:cluster_features=idx.cluster_features(cluster_id)forfeatureincluster_features[:10]:print(feature)# Perform a boolean query on the corpus, looking for documents that contain# both apples AND oranges in the text field.q=i[('text','apples')]&i[('text','oranges')]# Lookup all of the documents in the corpus that match this query.docs=idx.get_docs(q)# 'Pivot' the features in the index with respect to all cluster in the model.#  This will show the top 10 features in each cluster that are similar to the#  query.forcluster_detailinidx.pivot_clusters_by_query(query,top_k=10):print(cluster_detail)# This will show the top 10 features for a selected set of cluster_ids.forcluster_detailinidx.pivot_clusters_by_query(query,cluster_ids=[3,5,7],top_k=10):print(cluster_detail)DevelopmentInstallationTo install the development version:Clone the repository using git.From the cloned repository, use pip for an editable install:pip install -e .Running TestsThe full test suite and other checks are orchestrated via tox:python -m pip install -e .[test]

# To run just the testsuite
pytest

# To run everything, including code formatting via black and check coverage
tox