Predicting Electricity Distribution Feeder Failures
Using Machine Learning Susceptibility Analysis*
Philip Gross, Albert Boulanger, Marta Arias,David Waltz, Philip M. Long1,
Charles Lawson2, Roger Anderson

Matthew Koenig, Mark Mastrocinque,
William Fairechio, John A. Johnson,
Serena Lee, Frank Doherty, Arthur Kressner

Columbia University
Center for Computational Learning Systems
New York, NY 10115{philg | aboulanger | marta | waltz | plong |lawsonc | anderson}@ccls.columbia.edu

Consolidated Edison Company
of New YorkNew York, NY 10003{koenigm | mastrocinquem | fairechiow | johnsonj |leese | dohertyf | kressnera}@coned.com

1
2

Currently at Google Inc, Mountain View, CA.
Consultant, Digimedia Corp, Austerlitz, NY

Abstract
A Machine Learning (ML) System known as ROAMS(Ranker for Open-Auto Maintenance Scheduling) was
developed to create failure-susceptibility rankings foralmost one thousand 13.8kV-27kV energy distribution
feeder cables that supply electricity to the boroughs of New
York City. In Manhattan, rankings are updated every 20
minutes and displayed on distribution system operators’
screens. Additionally, a separate system makes seasonal
predictions of failure susceptibility. These feeder failures,
known as “Open Autos” or “O/As,” are a significant
maintenance problem. A year’s sustained research has led
to a system that demonstrates high accuracy: 75% of thefeeders that actually failed over the summer of 2005 were in
the 25% of feeders ranked as most at-risk. By the end of
the summer, the 100 most susceptible feeders as ranked by
the ML system were accounting for up to 40% of all O/As
that subsequently occurred each day. The system’salgorithm also identifies the factors underlying failures
which change over time and with varying conditions(especially temperature), providing insights into theoperating properties and failure causes in the feeder system.

Background

3. Primary Distribution: electricity is sent at mid-level
voltage (tens of thousands of volts) from substations tolocal transformers, over cables called feeders, usually
10-20 km long, and with a few tens of transformers per
feeder. Feeders are composed of many feeder sections
connected by joints and splices
4. Secondary Distribution: sends electricity at normalhousehold voltages from local transformers to individual
customers

Figure 1. Electrical Distribution

Electrical infrastructure has four main parts:
1. Generation: a prime mover, typically the force of water,
steam, or hot gasses on a turbine, spins anelectromagnet, generating large amounts of electricalcurrent at a generating station
2. Transmission: the current is sent at very high voltage(hundreds of thousands of volts) from the generating
station to substations closer to the customers

The distribution grid of New York City is organized into
networks, each composed of a substation, its attachedprimary feeders, and a secondary grid. The networks arelargely electrically isolated from each other, to limit thecascading of problems.
The feeders of the primary grid are critical and have asignificant failure rate (mean-time between failure of lessthan 400 days), and thus much of the daily work of theThis work has been partly supported by a research contract from Consolidated Edison.

Consolidated Edison Company of New York (Con Edison)field workforce involves the monitoring and maintenanceof primary feeders, as well as their speedy repair on
failure.
In the specific case of the underground distribution
network of Con Edison, transmission lines deliverelectricity into the city from upstate New York, New
Jersey and Long Island, as well as from in-city generation
facilities. Substations reduce the voltage to 33kV or less,and underground primary distribution feeders then locally
distribute the electricity to distribution transformers. From
there, the secondary network, operating at 120V/208V,
delivers electricity to customers. Our work focused on 941
underground primary feeders, distributing electricity to the
New York City boroughs of Manhattan, Brooklyn,
Queens, and the Bronx.

Problem
The underground distribution network effectively forms a
3-edge-connected graph – in other words, any two
components can fail without disrupting delivery ofelectricity to customers in a network. Most feeder failures
result in automatic isolation – called “Open Autos” or
O/As – and many more occur in the summer, especiallyduring heat waves when power use for air conditioning
adds to the load. When an O/A occurs, the load that had
been carried by the failed feeder must shift to adjacentfeeders, further stressing them. O/As put networks, control
centers, and field crews under considerable stress,
especially during the summer, and cost millions of dollars
in Operations and Maintenance (O&M) expenses annually.
One of Con Edison’s primary goals has always beenreliability, and its distribution network is the most reliablein the United States, and among the best in the world. The
goal is to maintain this standard of excellence even as
electricity consumption continues to grow. Over the years,
they have made enormous investments in manpower andmoney to collect and analyze vast amounts of data fromtheir systems. The ROAMS system described in this paper
is a continuation of Con Edison’s striving towards the mostreliable and efficient system possible, and it is built on thefoundation of the data and insights they have amassed overdecades.
We have initially focused on the specific problem ofranking primary distribution feeders according to theirsusceptibility to failure.
Con Edison has madeconsiderable efforts to identify, test and replace failureprone feeders in the fall and spring of each year, but
overall numbers of failures (normalized to weather
conditions) has remained fairly constant. A number ofpossible explanations have been offered for this: theincreasing loading and continuous aging of the grid areneutralizing the improvements from the maintenanceprogram; and/or the selection of reliability improvement
measures needs to be more effective.
The goal of the ROAMS system described here is to rank
the feeders most susceptible to impending failure with
sufficient accuracy so that timely preventive maintenancecan be taken on the right feeders at the right time. Con
Edison would like to reduce feeder failure rates in the most
cost effective manner possible. Scheduled maintenanceavoids risk, as work is done when loads are low, so thefeeders to which load is shifted continue to operate wellwithin their limits. Targeting preventive maintenance tothe most at-risk feeders offers huge potential benefits. In
addition, being able to predict incipient failures in close toreal-time can enable crews and operators to take short-term
preventative actions (e.g. shifting load to other, less loaded
feeders).
The work reported here is a key step in Con Edison’s longterm goal: system-wide evolution from reactive to
condition-based preventive maintenance processes andprocedures.

Application Description
We are not aware of previous AI-based systems applied to
power distribution problems. There have been a numberof efforts to improve the efficiency of complex systems byhaving a computer interpret a stream of sensor data.
However, these systems generally use human-constructed
expert or rule-based systems [7, 8, 9]. In contrast, we have
opted for a machine learning system that learns its models
entirely from data and hence does not include any humanDecision Support App
Decision
Support GUI

Feeder Load data

SQL
ServerDB

Xfmr Stress data

Outage data
ActionDriverActionTracker

Static data

Rankings

ML
EngineML
ModelsFigure 2: System Diagram. Incoming dynamic system data
at upper right is stored in the main database. The ML
Engine combines this with static data to generate andupdate models, and then uses these models to create
rankings, which can be displayed to the operator via thedecision support app. Any actions taken as a result are
tracked and stored back in the database.

biases. The overall structure of the ROAMS system is
illustrated in Figure 2.
A mixture of static data (e.g. age and composition of each
feeder section) and dynamic data (e.g. electrical load data
for a feeder and its transformers, accumulating at a rate ofseveral hundred megabytes per day) is combined into alarge data table. The data table currently has roughly 150
attributes for each feeder, and new ones are still being
added.
We then apply a boosting-based machine learningtechnique to the large data table, which outputs a learned
model. We evaluate these generated models by applying
them to subsequent test data using a performance metricthat captures how high actual feeder failures are positioned
in our ranked list of feeders (feeders predicted to have badsusceptibility will be higher in the list). More concretely,
our performance metric, which we refer to as thenormalized average rank of failures, is:1−∑ rank ( failure )inputs: list L of attribute-value descriptions of feeders withassociated nr. of failures; nr of boosting rounds T
output: marti-model M
1. let M be the empty model
2. for each round t=1,..,T do:- partition L into t sub-lists L1, .., Lt s.t. each Lj hassame nr. of failures; let th2, .., tht be the location of
the splits in terms of the normalized fraction offeeders that fall above the split.
- for each sub-list i=1,..,t do:i. compute quality of Li sortii. for each attribute A do:
1. sort Li according to A in ascending order,
compute quality of resulting sort
2. sort Li according to A in descending order,
compute quality of resulting sortii

# failures ∗ # feeders

For example, suppose that there are 1000 feeders in oursystem. If there are 3 failures on a given day, and thefeeders that correspond to these failures were ranked at 23,
65 and 147 in our list, then the resulting performance is:
1 – ( (23 + 65 + 147) / (3 * 1000) ) = 0.92
Note that a performance close to 1.0 (the ideal result)indicates that almost all actual outages were accounted forat the very top of our worst-feeders ranking. A
performance close to 0.0 (the worst possible result)indicates that all actual outages were at the bottom of ourranking. A performance of 0.5 would indicate that ourranking is no better than random.

t=1

t=2
(A2, asc)(A1, asc)(A5, desc)

Figure 3: An example of a Marti model with 2 levels (i.e.,
T=2): the first level (left) shows the attribute A1 with anascending sort direction; the second level (right) shows theother two attributes A2 and A5 that are part of the model.
Notice that the split of the two lists on which A2 and A5
operate is implicitly described by the height of the limitingline.

- if there exists attribute A and polarity P that improves
Li’s sort, then:i. if i > 1, add thi to M at level t, position i
ii. add A to M at level t, position i.
iii. sort Li according to (A,P)- else:i. if i > 1, add thi to M at level t, position i
ii. add “NOP” to M at level t, position i.
3. output M
Figure 4: MartiRank Algorithm.

When developing seasonal predictions used to guide
replacement planning, we gather large amounts ofhistorical data and train a few models based on different
random subsets of this data. The final ranking is computed
as the average of the rankings produced.
To create the control center operators’ near-real-time
display, we select a recent, well-performing model as the“current” model. Every 15 minutes, we apply this modelto the latest dynamic data, and generate a new ranking.
This is the first time that our new MartiRank algorithm,
described below, has been used for real-time operational
decisions.
An additional feature of this display is action tracking,which presents suggested actions that can be taken by fieldcrews to improve system reliability, and tracks these
operator actions. This tracking data is collected in a
database and is available as features for further machinelearning analyses, to evaluate the effectiveness of fieldcrew intervention. The optimal actions learned will be
used to design better preventive maintenance andoperational policies for managing the electric grid of New
York City.

Use of AI Technology
The core of the ROAMS system is a machine learning
ranking engine whose aim is to produce in real-time a listof the network’s feeders sorted from most to leastsusceptible to failure. To train our models, we use
Martingale Boosting [2], a recent variant of boosting.
Boosting [3] is a very successful machine learningtechnique that combines several “weak learners” or simple
classification rules, each with low individual accuracy, intoa powerful single model with high predictive accuracy. In
our case, the weak learners are functions defined by the
ranking produced by sorting on a single attribute. A highlevel description of our learning algorithm, MartiRank, isgiven in pseudocode in Figure 4. MartiRank greedily
selects in each of its rounds the attribute that is mostcorrelated with the failures in the given training data set.
The model records the selected attribute along with thedirection of its sort (ascending or descending), which wecall “polarity” in Figure 4. In round t (from t=1 up to t=T),
it splits the total data set into t sub-lists, on which it applies
its greedy attribute selection procedure; the list ispartitioned so that each sub-list contains the same numberof failures. Figure 3 shows an illustration of an example
Marti model.
We generate susceptibility rankings from a real-time feeder
description list and a trained model by repeatedly sorting
the list according to the attributes chosen by the trainedmodel. Intuitively, it is as if we walk the initial list
through the model from left to right, reshuffling the list
according to the attributes at that level of the model. Using
the example model shown in Figure 3, we first sort thewhole list according to A1 in ascending order. Then, we
split the list into two parts with the top part containing(roughly) 1/4 of the total number of failures and thebottom part containing the rest and we sort the top sub-list
following attribute A2 in ascending order and the bottom
list using the values of attribute A5 in descending order.
For the sake of comparison, we have been running twoother methods: SVM [4] and linear regression. For bothmethods we use the predictions for each feeder (thepredictions are real values) to get a final ranking by sortingon the predictions. As Figure 5 shows, MartiRank clearly
outperforms both methods. A further advantage of using
MartiRank over the other methods is that our models areeasily readable and interpretable by human users. This isabsolutely crucial for two reasons: (1) one of the main
goals of this application is to learn about root causes offailure so that preventive maintenance procedures can be
designed, and (2) operators using our system would nottrust a prediction system that they cannot interpret. Noticealso that MartiRank has been specifically designed to
produce ranking models, whereas most machine learning
methods need some kind of ad-hoc postprocessing (such as
the ones used with the regression methods mentioned
earlier) since the methods are not designed specifically for
ranking. Notable recent exceptions are RankBoost [5], and

RankNet [10]. Both of these methods address the problemof ranking by learning a real-valued function (used to rank
future test examples) from pairs of instances (x,y) s.t. x is
ranked higher than y in a given training dataset;
RankBoost [5] uses boosting and RankNet [10] uses neuralnetworks to learn the regression function. As no
implementation is available for these systems, comparison
to these alternatives is currently not feasible within thetime constraints. Moreover, RankBoost and RankNet are“black-box” methods, i.e., it is very difficult to interpretand understand the models they produce. Theinterpretability of its model and the excellent prediction
performance obtained so far have made MartiRank ideal
for this application.

Figure 5: This plot shows the performance of threecompeting methods (MartiRank, SVM, and linear regression)
during the second half of 2005. Each day, we compute eachalgorithm’s performance in terms of the normalized averagerank of failures that occurred during that day. MartiRankclearly outperforms the other methods. Further, the periods
that are critically important are the peak summer months,
when the system is near capacity and feeder failures are more
frequent, and these are the periods where MartiRank’s
accuracy is best. A histogram of daily failure counts is shown
below the line graph. All curves in the upper graph havebeen smoothed using a 5-day moving average for clarity ofpresentation.

Application UseControl center operators use a web-based interface to viewthe current ranking of feeder susceptibility. It shows thecurrent ranking, and highlights feeders that have changed
ranking since the last refresh 15 minutes ago.
Additionally, it integrates and displays useful dynamic datacollected in the process of assembling training data sets.
This includes information on how many of the feeder’ssections and transformers are operating close to theirlimits. The operator’s display is shown in Figure 6.
The susceptibility display is designed to work in
conjunction with three other existing applications, so
operators can take proactive actions based on worst case

Drill down on
this feeder

Figure 6: Operator’s display is used with 3 other web
applications to act on feeder susceptibility rankings.

analyses. Action tracking features have been added tothese existing applications, enabling the tracking ofactivities taken on susceptible feeders, and thus allowing
the quantification and measurement of resulting network
performance improvements. Two of these applications areshown in Figures 7 & 8.

Close open
network
protector
on V1629

Live Endcap
confirmed

Actiontracking

Figure 7: Action tracking is used to track the actions taken
on susceptible feeders to measure their effectivenessIn terms of prediction performance, we found that assummer 2005 became increasingly hotter, our ROAMS
System learned the mix of attributes most closely
correlated with impending failures, so that by July, wewere approaching 0.8 out of a possible 1.0 performance.
Figure 9 shows the ranks of feeders that failed on July 27th,a very hot day with an unusually-high number of failures(28 in total).3
During the month of July 2005, none of the 500 feeders
ranked lowest for susceptibility failed. More precisely, theworst ranked actual failure during the month of July was
442 (out of a possible of 941). There were a total of 184
failures during the month of July so that probability of thishappening by chance is less than 2-184.
We have a number of theories as to why performance wasbetter during the summer. The first is that many of the
input features to our machine learning algorithm weredeveloped by Con Edison with a specific focus onmodeling the electric distribution system during heatwaves. The second is that distribution system failures may
have more deterministic causes during heat waves, as theload and stress contribute directly to cable, joint, andtransformer problems, while in the cooler months, failures
tend to be more random and difficult to model. However,these are just conjectures, and we are continuing toinvestigate this question.

3

Complete statistics with results for the summer can be found at
http://www.cs.columbia.edu/~marta/roger

Switch check
Record added

Figure 8: To close the network protector on the transformer invault V1629, the Switch Check application is used to open a
job to dispatch crew to close the switch. Action Tracking
records that the operator confirmed that he added a newswitch check job to close the network protector.

Payoffs
Reacting to an unscheduled feeder failure has a significant
cost in manpower and materials, as well as potential risk ofcustomer power outages. Our ROAMS system is helping
to mitigate those consequences in a number of ways.
Our long-term recommendations of feeder susceptibility
are helping to decide which feeder components should be
replaced during the off-season. Although the results will
only become apparent over the coming years, we hope to
see failure rates dropping again, resulting in a system that
operates in a more cost effective manner.

Development, Deployment, and Maintenance
There were a number of development challenges that had
to be overcome before the system could be deployed. Con
Edison is a large company, and its data is spread overmany repositories. Simply identifying the data that would
be useful for the system and then arranging accessmechanisms took substantial time (and is still continuing).
As Con Edison has always been active in using advanced
technology, many of their software systems have been in
use for a considerable amount of time, and retain their
original orientation towards human-readable output. We
wrote programs, sometimes of substantial complexity, toconvert this data back into a computer-usable form.
Figure 9: Failed feeders of 27th July 2005. The x axis
shows the rank of the failures, the y axis shows the
cumulative number of failures up to that rank.

Operators are using the short-term prediction capabilitiesof the current system in a number of ways. In particular,
they can now augment their existing information about thestructural risks of a failure, i.e. what loads will increase
and by how much due to the loss of the feeder, with anestimate of the likely risks of a failure, i.e. which feeders
are actually most likely to fail under the newcircumstances.
Con Edison is investigating designs for a new generationof equipment that will allow dynamic switching of current
flows. We are working together to develop new machinelearning techniques integrated with system-wide models sothat as this new equipment comes online, more advanced
and proactive responses will be possible for risks identified
by our ROAMS System.

A later challenge was the volume of data. Our software
processes were initially unable to keep up with thegigabytes of data being accumulated on a daily basis. A
full reorganization of our underlying database tablestructure and the addition of a preprocessing stage solved
the problem.
Having solved these problems, screens displaying thecontinuously-updated failure susceptibility rankings forfeeders have been in use since mid-2005, about sevenmonths prior to writing this paper. Failure susceptibility
rankings to support maintenance scheduling are being usedwith Fall 2005/Winter 2006 maintenance planning.
Results from use of our ROAMS System to-date havegiven Con Edison the confidence to incorporate ourfindings into their current preventive maintenance program
and to continue the development and deployment of thistool in 2006.

The most impressive benefit from our work so far hascome from an analysis of the preventive maintenance thatCon Edison already does on its feeders. Con Edison has a
program to ensure the reliability of its feeders through an
in-field test. High DC voltage is put across a feeder for asustained period (the tests are known as “Hi-Pots”), andany resulting faults are immediately fixed. The tests are
effective for “shaking out” latent problems, but are
expensive and can put substantial stress on the feeder.
We discovered that our susceptibility ranking wasextremely good at predicting which feeders were likely tofail their Hi-Pot. More importantly, those “good feeders” at
the low-risk end of our ranking were virtually guaranteedto pass the Hi-Pot, and could safely be skipped.
Con Edison is now including our susceptibility ranking intheir evaluation of candidate feeders for Hi-Pot tests thisseason, and we expect to see substantial savings from a
reduction in unnecessary tests, as well as increased
reliability from feeders that are tested and pass.

Figure 10: Feature categories ranked in the top five of thefirst pass of MartiRank (below) are leading indicators ofactual feeder failure causes (above). Categories are shownstacked by type.

Future Work
Within the machine learning module there exist several
directions for improvement. Our system uses, at any given
time, the most recently trained model to make its
predictions. It is currently up to the operators to decide
when to re-train a model if they observe that performancedegenerates. We are currently designing a dynamic online
algorithm that will select optimal or quasi-optimal models
based on recent past performance of competing models.
This next-generation ML engine aims for full automationof the system and removal of assumptions that might behurting the current performance. Further, it may allow for
an interesting knowledge discovery process, as we learn torelate environment and system states to optimal modelcharacteristics.
Additionally, we are starting to investigate the use of ourML system to identify root causes for failures. The firstpass of the MartiRank algorithm evaluates each featureindividually for its ability to predict outages across theentire set of feeder training data. We have found that if wegroup these features into general categories (electricalcharacteristics, transformer stress, cable type, etc.), the topranked feature categories from the first MartiRank pass are
effective leading indicators of corresponding actual failurecauses, with a lead time of around 6 days.
For example, we see a rise in Hi-Pot test related features in
our analysis around a week before seeing a corresponding
rise in actual feeder failures caused by burnouts, as shownin Figure 10.
We believe that further study of the relationship between
ML-identified attributes and actual causes may lead tofurther improvements in fault management processes andsystem reliability.
A related prediction problem that arises in the context ofour collaboration with Con Edison is that of making
quantitative predictions about the time to failure of each
feeder and relevant component in the network. This very
hard problem could potentially be solved by a mixture ofmachine learning and statistical techniques such as
survival analysis [6]. Although we are not currently
focusing on this line of research it is part of our longerterm research plans.

Acknowledgements
Too many people at Consolidated Edison contributed tothe success of this project to name individually. We wish
to thank all of those who gave so generously in time,
advice, and effort. We would particularly like to recognize
the following: MECC Operations: Matt Sniffen;
Distribution Engineering: Elie Chebli; Cable and Splice
Center of Excellence: George Murray, Bill McGarrigle,

and the Van Nest team; Statistical Consulting: Dr. David J.
Allen.
Additionally, a number of Columbia faculty, students,
staff, and researchers contributed towards this effort:
Rocco Servedio, Samantha Cook, Hila Becker, Samit Jain,
John Ioannidis, and Sergey Sigelman.

References[1] Hanley, J. A. and B. J. McNeil. The meaning and use of thearea under a receiver operating characteristic (ROC) curve.
Radiology 143: 29-36, 1982.
[2] P. Long and R. Servedio. Martingale Boosting. Eighteenth
Annual Conference on Computational Learning Theory (COLT),
2005, pp. 79-94.
[3] Y. Freund and R. Schapire. A decision-theoretic
generalization of on-line learning and an application to boosting.
Journal of Computer and System Sciences, 55(1):119–139, 1997.
[4] N. Cristianini and J. Shawe-Taylor, An Introduction to
Support Vector Machines and other kernel-based learning
methods. Cambridge University Press, 2000.
[5] Y. Freund, R. Iyer, R. E. Schapire, and Y. Singer. An efficient
boosting algorithm for combining preferences. Journal of
Machine Learning Research, 4:933–969, Nov. 2003.
[6] D. Kleinbaum and M. Klein. Survival Analysis: A Selflearning Text. Springer, 1997.
[7] Rob Wylie, Robert Orchard, Michael Halasz, François Dubé.
IDS: Improving Aircraft Fleet Maintenance. AAAI/IAAI 1997,
pp. 1078-1085.
[8] R. Helfman, E. H. Baur, J. Dumer, T. Hanratty and H.
Ingham. Turbine Engine Diagnostics (TED): An Expert
Diagnostic System for the M1 Abrams Turbine Engine.
AAAI/IAAI 1998, pp. 1032-1038.
[9] G. S. Semmel, S. R. Davis, K. W. Leucht, D. A. Rowe, K. E.
Smith and Ladislau Bölöni. NESTA: NASA Engineering Shuttle
Telemetry Agent. AAAI 2005, pp. 1491-1498.
[10] C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N.
Hamilton and G. Hullender. Learning to Rank using Gradient
Descent. Proceedings of the 22nd International Conference on
Machine Learning (ICML), 2005, pp. 89 - 96.