Handling imbalanced data using Geometric SMOTE
By: Vivek Vinushanth Christopher
Tags: Machine Learning, Imbalanced Data, Programming, Smote, Computer Science

Algorithm

Please do refer to the G-SMOTE paper for the complete algorithm and explanation. Here, I wish to explain the algorithm with examples and abstract out.

1. The Smin elements are shuffled and an empty set Sgen is initialized

Smaj = {smaj1,smaj2,smaj3,.......}

Smin = {smin1,smin2,smin3,........} -> after being shuffled

Sgen = {}

2. Repeat until N minority instances are generated in Sgen:

N = 0

- Have to run the procedure till N number of samples are generated

2.1. Let x_center ∈ Smin the selected minority class instance of p components.

- A minority point from Smin is chosen as the center of a geometric region

x_center = smin1 - The order of choosing the minority point will be the order of shuffled data points and in case of N > size of Smin, continue to re-run from the beginning again.

2.2. If α_sel = minority:

In G-SMOTE algorithm they defined 3 neighbor selection strategies ; minority,majority and mixed. - Find the k nearest neighbors of x_center from the set of Smin

k_nearest_neighbors = {smin6,smin5......} - randomly select x_surface from the k_nearest_neighbors

x_surface = smin5 - Calculate a radius from the relation

R ← d(x_center, x_surface)

2.3. If α_sel = majority:

- choose the nearest majority neighbor (from Smaj) of x_center

x_surface = smaj3 - Calculate a radius from the relation

R ← d(x_center, x_surface)

2.4. If α_sel = mixed:

- Find the k nearest neighbors of x_center from Smin.

k_nearest_neighbors = {smin6,smin5......} - randomly choose one from k_nearest_neighbors

smin6 - Calculate the euclidean distance

dmin ← d(x_center, smin6) ---------------------------------------------------------------- - Find nearest majority neighbor of x_center from Smaj

xmaj= smaj3 - Calculate the euclidean distance

dmaj ← d(x_center, xmaj) ---------------------------------------------------------------- - Calculate a radius from the relation

x_surface ← argmin (dmin, dmaj)

R ← d(x_center, x_surface) ----------------------------------------------------------------

Note: Since d ≤ R ≤ dmaj; where d is distance from x_center to the generated sample

=> Generation of noisy samples is avoided

2.5. Generate a synthetic sample x_gen ← hyperball(center=0,radius=1)

Fig. 7. Generating e_sphere

hyberball: - function is meant for creating the e-sphere - e_sphere is generated on the surface of a unit hyper-sphere

2.6. Transform1 the synthetic sample by x_gen ← truncate(x_gen, x_center, x_surface, α_trunc).

Fig.8. Truncate effect of α_trunc

Truncation: - defined with bit of complex mathematics. But intuitively tells us which side of the hyper-sphere to be truncated to secure a safe space to generate a minority sample. - When α_trunc > 0, the area that is opposite to the x_surface is truncated from the interior of the hyper-sphere. - When α_trunc < 0, the truncation occurs in the area that includes the x_surface point. - α_trunc value is generally -1 to 1. If it is -1.0, it truncates the exact semi-hyper sphere on the same side of surface point and truncates the opposite side of semi-hyper sphere if α_trunc equals 1.0

2.7. Transform2 the synthetic sample by x_gen ← deform(xgen, xcenter, xsurface, α_def ).

Fig.9. Effect of deformation with α_def

Deformation:

- corresponds to the deformation of the hyper-sphere in to a hyper-spheroid - α_def relates the story of hyper-plane that to be chosen on which the new minority sample will be synthesized - α_def typically takes value 0 to 1 allowing the Deformation method to choose the proper plane to place generated sample.

2.8. Transform3 the synthetic sample by x_gen ← translate(xgen, xcenter, R).

Translation: - translation of the generated point by the x_center vector and the resealing by the value of the radius R - return (x_center + R * x_gen)

Fig.10. Generation of minority sample

2.9. Add the generated sample x_gen to the set of generated samples S_gen.