Mixed Methods for Survey DataAnalysis
Data Preparation and Importing:

Here is an example Excel worksheet after editing to prepare its data for import into a
MAXQDA project.
• Insert Column A which controls the grouping of the texts in MAXQDA’s“Document System” window. Enter the same phrase for all rows if you want all
texts to be in a single group, here “Survey”.
• Column B contains the labels for the Document System in MAXQDA, it shouldhave a unique identifier for each respondent. This is probably an output from yoursurvey program.
• The labels at the top of each column (Row 1) will appear exactly like this in
MAXQDA, so use labels that will be meaningful in that different context.
Remember that shorter labels are better than long ones,• You may edit these labels, but you should not edit the response cells.
• If you use Survey Monkey for data collection, there is a special routine in
MAXQDA to collect the data from your Survey Monkey account which by-passes
this step. Full instructions are available in the MAXQDA Online Manual.
• When your data is ready, close the Excel workbook, open the analysis project inMAXQDA and use the option Import > Survey Data > Import Data from Excel
Spreadsheet. You will need to navigate to the folder with the Excel workbook
and select it there to start the process.

- Use the pull-down menus toselect the column headings forColumns A & B (see above).

- I have scrolled down the list
to show the 2 open questionsin this data ticked in the "Code"column.

- I recommend leaving thisunticked.

• This is where you separate variables from qualitative texts. The texts to beanalysed have ticks in the “Code” column, the short or quantitative data go into
the “Variable” column.
• MAXQDA tries to do this for you and most of its ticks will be correct, but you doneed to check them all (this screenshot was taken before any editing was done).
• I recommend leaving the option “Code empty cells” box blank (as above) because
then you get a simple count of all collected responses for each question and nonresponses are ignored.
• A similar version of this dialog appears as part of the special routine for importing
data from Survey Monkey.
When you click the "OK" button the program moves on to setting the formats for theVariable data when they are imported into your project.

- this is a pure number, so Integer
is correct.
- a range of numbers like this has
to be processed as text.

Again, MAXQDA offers its best guesses for the middle column, based on the data inthe second row of the spreadsheet. You can edit these with the pull-down menu
options for each row in this table. Only the items ticked as "Variable" in the previous
dialog are involved in this stage. You may now regret the way you collected some ofyour data as "Text" is the only practical option for several numerically based fields.
Click on "Import" to set the function running.

- When the import has completed take theopportunity to copy this summary and paste it into a
Free Memo or the project Logbook as a record of thisprocess.

The final step in the importing stage is to check that what has arrived in your projectis what you expected.

•I have selected one document (RESP43) in the Document System to display
its responses to both open questions in the Document Browser.

•I have also opened the Variables > Data Editor for Document Variables andscrolled that to show the same case in the lower window above.

•The Code System is showing the code names for the 2 open questions, andthese can also be seen applied to the responses in the Document System.

•At this stage it is a good idea to paste the full question text into the Code
Memo for each of the open questions for easy reference during analysis.

•The numbers beside each code in the Code System show how many
responses were found for each question (this would not be correct if youticked "Code empty cells" during the import).

•Once you have checked that all of the data you expected has been imported
successfully, you are ready to begin the analysis.

Coding the open question responses:
The next step is to apply codes to the text data that is stored in the Document
System, ie the responses to your open-ended questions. You have a wide spectrumof ways to work with this data from fully manual (reading one at a time, deciding
what codes are relevant, and applying those codes by drag and drop), through avariety of semi-automatic tools to asking AI Assist to help. This session will not lookat these in great detail, there are other recorded webinars and sessions you canaccess for that, but we will just look at some of them briefly.
You should consider the analysis methodology you want to use, the consequences of
the way you collected the responses, the objectives of your research questions, andthe quality of the data you are using to inform your decisions about how to handle
this task. Some experimental work with a small sub-sample of your data may be
useful before you commit to your chosen approach, to make sure it will work.
A good place to start is the function Analysis > Categorize Survey Data .

I activated all the Survey documents and dragged the code for the first open
question into the setup dialog to get the above screen. Here you can now work inmany different ways with the convenience that you can easily view all the segments
already attached to any particular code with a single click in the left section.

You can create codes manually in the left section - it is useful to make all thethematic codes derived from one question's responses as sub-codes of thatquestion's code as this makes reporting the basic statistics easy. You can dragsegments onto those thematic codes within this window, and also change or edit
any of these decisions.
With the Word Cloud function in the toolbar, you have access to word frequency
tools to start identifying common themes on that basis. With the Search & Autocode
function you can partially automate the process and allocate codes to multiple
responses which share the use of key words. With the Dictionary function you cantake this a step further and store multiple search and autocode commands into asingle step (this is useful if you plan to repeat a survey in successive waves orrounds of data collection). Or you can use AI Assist from this toolbar to summarise
the responses or to suggest multiple subcodes for you. In all probability, you will
use several of these options, including manual corrections of the segments which
use unusual language or mislead the automated tools. We will look at the AI Assist
functions which are the newest innovations in this area.

- The bullet list with examples will help
you to apply those suggestions which you
choose to adopt.
- The upload tick is required for thisfunction to work. Please check the FAQs tounderstand the implications of this.

Below is an extract from the Code Memo, which is where the suggestions are put.
You may find it useful to run the same request a couple more times, as AI Assist may
come up with some slightly different suggestions, giving you more choice.

- The numbered line is asuggested code name, followed by
3 content examples which may be
useful for Search & Autocodeprocesses.

- There were 7 suggestions fromthis query.

You can easily see several areas of overlap within these suggestions, so it would be
difficult to apply all of them in this exact structure. However, you may find thatseveral fit quite well with your main research questions and so it would be worth
using those as the next step. For illustration, I will use the first two.
My approach is as follows:•Create the selected new codes as sub-codes of the question code.

•Use copy & paste to transfer the content examples into their respective code
memos.

•With the main question code selected (so that you have all of its responses inthe window) explore Search & Autocode to attach the most common ideas tocodes.

•With this kind of data it is generally best to apply the thematic codes to thewhole response each time (rather than just keywords or sentences).

•Manually check and edit the results, including adding codes to relevant
segments which used different words. Remember that some responses may
be linked to several subcodes, but every response should be linked to at least
one subcode.

Note that, when you use this autocode function, you cannot directly autocode into
your existing codes as you have to autocode into a code with the name of the words
you searched on. But then, within this function window, it is quick to use the context
menus to move all of the newly coded segments from the temporary autocode to
the desired code and delete the unwanted autocode.
You may find it useful to practise using these functions on some unimportant data(such as the example projects included with MAXQDA) before using them on yourreal data. Ideally, you want to be able to add, edit and remove codings while
thinking more about the meanings in the data than the processes in the software.

Using data from both open and closed question
responses:
This is the point at which the benefit of importing the variable data as well as theopen questions becomes apparent. If your survey sample is large enough andsufficiently randomly selected, then you may reasonably make inferences from howvarious subgroups of your cases raise some of the themes you identify in the coded
responses. This can be done quite easily, either as quantitative frequency tables or,more excitingly, with informative text quotations to show differing meanings.
First, let us look at the basic reporting of the frequencies of the codes we have justbeen applying to the open questions. This can be done simply with the function
Analysis > Code Frequencies. It is best to exclude the overall frequency of theresponses to the question itself, so activate all the Survey documents and the wholegroup of analysis codes for the question being examined before starting thefunction. The dialog is then easy to complete by clicking on the box for "Insert
activated codes" to bring them all in and then use the "X" button on the row for thequestion code itself to exclude that and leave all the thematic codes. The dialog
should default to tick the box for "Only for activated documents" which is what youwant.
In the table which opens, you can choose the sequence in which to display theresults, alphabetical order of the code names or descending frequency of theresponses. Then click on the "Chart view" button (2nd on the toolbar). I find that thehorizontal bar chart is useful for this kind of display because the code names areclearer.

In the screenshot above, I have circled the pair of icons in the top left corner whichswitch back and forth between table view and chart view. On the right side I havecircled the icons which switch between horizontal and vertical bar charts. Other
toolbar icons allow you to add or remove percentages and frequencies, changecolour schemes and edit titles to allow you to create the visual appearance youwant.
To start looking at interactions between data from closed questions and these
themes we can use the function Mixed Methods > Crosstab. We can use the sameactivations as before. The dialog to set this up is shown below. I first ticked the box(circled) at the bottom left for "Insert all variable values as conditions", then selected
the variable I want to use - this time "Age" - and finally clicked on the arrow in the
middle of the dialog to move "Age" and all of its values into the middle part of thedialog to become the "columns" in the table. The program has defaulted to add ticks
on the right for only activated documents and codes, because it detected that someof each had been activated.

Below is the table generated from this dialog:

I have clicked on the 2 circled icons. The first of these reduced the width of the tableby reducing the information shown in each column header, I would need to makesure that any subsequent presentation of this table clearly states that the columns
refer to the ages of the respondents. The second has set the display to "Rowpercentages".
We need to be careful in this type of table to account for the fact that different
numbers of respondents fall into each age-range used, so raw frequency numbers
can be misleading. The top row of data shows the percentages for each age groupthat responded to this question, the bottom row of data ("# N = Documents") shows
the percentages for each group of participants in the survey some of whom did not
respond here. Where a percentage in the rest of the table is very different from bothof these it might be an indication of something interesting. The sample size here isnot sufficient for these differences to be reliable but, hopefully, you will collect moredata and find useful results in this way.
The final step is to convert the frequency table into a quotations table. This is doneby clicking on the first icon in the toolbar of the Crosstab table ("Segment Matrix").

This table displays all of the source texts behind the Crosstab table. By clicking on acode in the first column, you display all of its segments in the table. Scroll to theright with the slider at the bottom of the screen to see the columns for older agegroups. Each column of texts is independently scrollable (see the vertical slider on
the "30-39" column). The circled toolbar icon toggles the display of Document
Names on/off. In this table you can consider how the language used by respondents
of different ages differs for any of the themes identified in your coding, which can be
more interesting than simple frequency differences.

Graham Hughes (n.graham.hughes@ntlworld.com)

September 2024