Why Annotation Matters:

Semi-Structuring MWO Text to Inform 

Fault Detection Methods

04/12/2021

Anna Conte

Coline Bolland

Lynn Phan
Overview of Nestor
Nestor

3

● An NLP annotation software tagging system 

that performs structured data extraction from 

the MWO free text fields

● Allows user to tag individual words (tokens) 

from unstructured text

● Tokens can be aliased in order to handle 

abbreviations and misspellings ○ Usually, MWO text needs to be 

grammatically perfect for themes to be 

detected

○ Problem avoided: misspelled words 

would be grouped with correctly spelled 

words under the same theme if they are 

related

● Classifications: S (Solution), P (Problem), I 

(Item), SI (Solution Item), PI (Problem Item)
Overview of Use Case
Use Case Summary

• Purpose

• Help HVAC researchers determine sensor placements using data from historic 

MWOs

• Maintenance Work Order Dataset

• Mixed use of laboratory and office space

• Combination of scheduled maintenance, ad hoc maintenance, and occupantsubmitted requests (e.g. temperature complaints)

• Approach

• Focus: 1 building

• Use KPI calculations (with Nestor tags) to identify assets that may cause the 

most problems or need more attention

• → Placing sensors on these machines would help HVAC researchers collect the 

most data about faults

5
Overview of Data

• 405 rows for this building

• Ten years (2009-2019)

• Missing data:

• 87% missing “ASSETNUM” 

• Most frequent tag: “set set point” (64.6% among the tags ) for the “SI Solution Item” 

tag

• Large portion of the dataset is related to adjusting temperature setpoints, not 

necessarily mechanical issues

6
First Attempt to Find Worst Asset: Use MWO Counts

7

● Try to overcome missing Asset 

Number data with text extraction

● Using text extraction to gather 

asset numbers from the free text 

fields, we were able to increase 

from 13% to 22% of MWOs with 

an asset number, but this is still 

only a portion of the data.

● So, we need a different approach

Top Ten Assets, by MWO Count
Use MWO Counts: By Location

● “LOCATION” is a more complete 

data field

● About 100 MWOs are simply labeled 

“Building A”, but that still leaves 

about 75% of the dataset with 

useful information

● The first and second floors have 

the most MWOs. Assuming that 

these are the more occupied floors, 

they are likely to also have an 

inflated amount of MWOs due to 

comfort/temperature complaints.

8
Using Tags to Categorize MWOs

● If MWO includes 

“too_hot”, “too_cold”, 

and/or “request”, it is a 

“comfort” MWO

● Room 4 has large amount 

of complaints, but they 

are not comfort-related 

(mechanical space)

9
Using Tags to Categorize MWOs

➔ Other ways to apply 

categorization with tags:

● Planned vs. Unplanned 

Maintenance

○ “Monthly”, “Weekly”, 

“Daily”

● Specific tags

○ “Replace”, “Repair”

10
Room 4 (Mechanical Room)

Items

● Water

● Water_detection

● Air_conditioning_unit

● Detection

● Detection_alarm

● Humidity

● Valve

11
Worst “Topics” by Cost

Costliest (meaningful) tags from 

Nestor were:

● Set point

● Water leak

● Reset alarm

● Replace valve

● Replace belt

● Cold building

● Hot room

Assets relating to these tags may 

be the most “problem-causing” 

(costliest to address)

12

Building A: Top 15 Costliest Tags
Worst “Topics” by Cost

➔ Nestor tags allow 

opportunities for data 

values to be examined in 

terms of concepts:

◆ Cost

◆ Average MWO 

duration

◆ Average amount of 

time behind schedule

13

Building A: Top 15 Costliest Tags
Examine “Topics” by Time

● Structured text allows to analyze topics

relative to time

● Survival analysis to observe how long work

orders for different problems remain open

● MWOs for complaints of “too hot” are

typically addressed more quickly than

complains of “too cold”

● Same logic between “Replace valve” and

“Reset alarm”

14

“too_hot building”

“too_cold building”

“Replace valve”

“Reset alarm”
Worst Assets

Based on high volume of MWOs for Floor 1, whichever unit in Room 4 (Mechanical Room) is 

serving the first floor is likely the best candidate for “worst asset”.

15

AHU-1

● Water_detection alarm

● Fan

● Freeze_thermostat

● Discharge

● Valve

● Chilled_water

● Thermostat alarm

● Alarm reset

● Drain

● Freeze

● Set_point

AHU-2 

● Water_detection alarm

● Fan

● Controller_valve

● Alarm

● Air leak

● Cleaned

AHU-3 

● Water_detection

● Damper

● Actuator

● Diaphragm

● Exhaust

● Hood

● Exhaust_fan

● Fume_hood

● Leak

● Set_point
● Similar analyses have been done for other facilities (manufacturing)

● Our upcoming paper (under review) examines effects of missing data

○ When data quality varies based on the context of a MWO, the resulting calculated 

KPIs can become very inaccurate

○ By structuring text data, we can understand these patterns

Other Applications

16

Quantifying Some Effects of Missing Data on KPI Calculations
● By semi-structuring the free-text fields, we can make greater sense of what is 

going on within the facility, despite having a less-than-ideal dataset

○ Highlight types of issues that arise in a facility

○ Understand data quality patterns within dataset

● Analyses involving semi-structuring of raw text can be applied to a variety of 

different contexts

○ Maintenance (HVAC, Manufacturing)

● Using Nestor tags offers more insight into MWO data than analyzing raw text 

and numerical data

Key Takeaways

17