Reproducible Intelligent Pigging Data in R Language
By: Purnomo Setyawendha
Tags: Pipeline Integrity, R

R Series for Pipeline Integrity Engineer

Photo by Brett Jordan on Unsplash

Why we need Reproducible Intelligent Pigging Data ?

When you asking for help in a community platform such as stackoverflow, it is very important for you to be able to describe your problem through a minimal, reproducible example (reprex), a minimal, complete and verifiable example (mcve), or a minimal, workable example (mwe).

As the real Intelligent Pigging data are confidential information that cannot be shared in public, therefore it is become more important to generate dummy toy datasheet as object that can represent the real data to test various methods of analysis, visualization, modeling, prediction or other machine learning method and to be shared in open environment such as stackoverflow to discuss about an issue or sharing the solution.

Typical Intelligent Pigging Data

Intelligent Pigging Data can be describe as a table for list of anomalies/defects detail that identified by Intelligent Pigging tools.

Pipeline Defects Properties (by Author)

Following data structure consider as minimum data to represent general Intelligent Pigging Data :

Log Distance, is defect/anomalies location by distance between the defect to reference point (Log Distance =0)in float number in meter unit Orientation, is defect/anomalies circumferential location in clock position as text/string. There is a convention in Inspection domain that circumferential position describe as clock position of 00:00 to 12:00 instead of 0–360⁰ angle, depend on data input process and format, in worse scenario it will be treated as string. Wall Thickness is specific pipeline wall thickness on the defect location, represent by float number in mm Wall Loss is percentage ratio of defect depth over pipeline wall thickness represent by integer due to most tools accuracy limitation. Defect Length is the length of defect along pipeline direction in mm represent by integer due to most tools accuracy limitation. Defect Width is the width of defect across pipeline direction in mm represent by integer due to most tools accuracy limitation. Surface is defect location either at Internal or External Surface, represented as string as raw data, on analysis this should be convert to binary / Boolean, value of true or false, or 1 and 0.

Pipeline Defect Dimension (by Author)

Input Data Requirement

Some input are required to generate defects list :

Pipeline Length in meters, the defects later will be generated uniformly across pipeline length Number of defects that will be generated Ratio of internal over external defects, this only optional, and we can put 1 for equal number of internal and external defects

In principle all defect are uniform randomly generated within the normal range for typical general corrosion.

Pipeline may designed in multiple sections and each section may have different wall thickness.

Pipeline Sectioning (by Author)

Therefore as input all wall thickness (WT1, WT2, WT3…..WTn, WTn+1) and wall thickness change limit (Limit1, Limit2, Limit3….Limitn) need to be defined. Limit0 (starting point) and Limit n+1 not require input as they already defined as starting point with Log Distance = 0 and end point with Log Distance = pipeline length.

R Coding

Input Value for Number of Defects, Pipeline Length (m) and Internal/External ratio

NumberofDefects <- 300 #Input for Number of Defects Generated

PipelineLength <- 20000 #Input for Pipeline Length in meters

IntExtRatio <- 5 #Input for Ratio between Internal/External Defects

Defect Log Distance using uniform sample function

LogDistance <- sort(sample(1:PipelineLength,NumberofDefects, replace = T))

Defect Width (mm) , Wall Loss (%), and Defect Length (mm) generated using random exponential function

DefectWidth <- rexp(NumberofDefects,0.2)

WallLoss <- rexp(NumberofDefects,0.1)

DefectLength <- rexp(NumberofDefects,0.075)

Orientation generated using sample as string, this to simulate data provided from the Intelligent Report where orientation not classified as time series data

d <- paste('0',1:9, sep='')

H <- sample(c(d,10:11),NumberofDefects,replace = T)

M <- sample(c(d,10:59),NumberofDefects,replace = T)

Orientation <- paste(H,":",M, sep = "")

Wall Thickness can be generated in few options, this time I choose to use combination of ifelse and outer to avoid multiple for loops

# Range of Wall Thickness Values as c(WT1, WT2, WT3....WTn)

WTValues <- c(24,20,22,20,24) #5 arguments # WT Limit or Location Where WT change c(L1, L2.....Ln-1)

WTLimit <- c(500,9750,10250,19500) #4 arguments LimitCheck <- function(a,b){

ifelse(a>b,1,0)

} WT <- function(PLength,Limit,WT){

Matrix <- outer(PLength,Limit,"LimitCheck")

Section <- rowSums(Matrix)+1

return(WT[Section])

} WallThickness <- WT(LogDistance, WTLimit, WTValues)

After all parameter have been generated as lists, we combine them all together as dataframe

IPData <- data.frame(LogDistance, Orientation, WallThickness, WallLoss, DefectLength, DefectWidth, Surface)

Due to tools accuracy limitation in the real world, Defect Length, Defect Width and Wall Loss mostly only deliver as integer, therefore to simulate this we round up all of these parameters (optional)

IPData$DefectLength <- ceiling(IPData$DefectLength)

IPData$WallLoss <- ceiling(IPData$WallLoss)

IPData$DefectWidth <- ceiling(IPData$DefectWidth)

Here you go

The first 10 rows of IPData DataFrame (by Author)

You can also write the IPData Dataframe into csv file to be shared or modify in excel