Repository: xataio/pgstream
Pull request #155 (2025-02-21 13:08:04)

Add gm float transformer
Introduce `FloatTransformer` which uses `greenmasktransformers.RandomFloat64Transformer` under the hood.

github-actions[bot]: ### Merging this branch changes the coverage (1 decrease, 1 increase)

| Impacted Packages | Coverage Δ | :robot: |
|-------------------|------------|---------|
| github.com/xataio/pgstream/pkg/transformers | 100.00% (ø) |  |
| github.com/xataio/pgstream/pkg/transformers/builder | 16.67% (**-2.38%**) | :thumbsdown: |
| github.com/xataio/pgstream/pkg/transformers/greenmask | 94.06% (**+1.87%**) | :thumbsup: |

---

<details>

<summary>Coverage by file</summary>

### Changed files (no unit tests)

| Changed File | Coverage Δ | Total | Covered | Missed | :robot: |
|--------------|------------|-------|---------|--------|---------|
| github.com/xataio/pgstream/pkg/transformers/builder/transformer_builder.go | 16.67% (**-2.38%**) | 24 (+3) | 4 | 20 (+3) | :thumbsdown: |
| github.com/xataio/pgstream/pkg/transformers/greenmask/greenmask_float_transformer.go | 94.59% (**+94.59%**) | 37 (+37) | 35 (+35) | 2 (+2) | :star2: |
| github.com/xataio/pgstream/pkg/transformers/transformer.go | 100.00% (ø) | 7 | 7 | 0 |  |

_Please note that the "Total", "Covered", and "Missed" counts above refer to ***code statements*** instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code._

### Changed unit test files

- github.com/xataio/pgstream/pkg/transformers/greenmask/greenmask_float_transformer_test.go

</details>

github-actions[bot]: ### Merging this branch changes the coverage (1 decrease, 1 increase)

| Impacted Packages | Coverage Δ | :robot: |
|-------------------|------------|---------|
| github.com/xataio/pgstream/pkg/transformers | 100.00% (ø) |  |
| github.com/xataio/pgstream/pkg/transformers/builder | 16.67% (**-2.38%**) | :thumbsdown: |
| github.com/xataio/pgstream/pkg/transformers/greenmask | 94.06% (**+1.87%**) | :thumbsup: |

---

<details>

<summary>Coverage by file</summary>

### Changed files (no unit tests)

| Changed File | Coverage Δ | Total | Covered | Missed | :robot: |
|--------------|------------|-------|---------|--------|---------|
| github.com/xataio/pgstream/pkg/transformers/builder/transformer_builder.go | 16.67% (**-2.38%**) | 24 (+3) | 4 | 20 (+3) | :thumbsdown: |
| github.com/xataio/pgstream/pkg/transformers/greenmask/greenmask_float_transformer.go | 94.59% (**+94.59%**) | 37 (+37) | 35 (+35) | 2 (+2) | :star2: |
| github.com/xataio/pgstream/pkg/transformers/transformer.go | 100.00% (ø) | 7 | 7 | 0 |  |

_Please note that the "Total", "Covered", and "Missed" counts above refer to ***code statements*** instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code._

### Changed unit test files

- github.com/xataio/pgstream/pkg/transformers/greenmask/greenmask_float_transformer_test.go

</details>

github-actions[bot]: ### Merging this branch changes the coverage (1 decrease, 1 increase)

| Impacted Packages | Coverage Δ | :robot: |
|-------------------|------------|---------|
| github.com/xataio/pgstream/pkg/transformers | 100.00% (ø) |  |
| github.com/xataio/pgstream/pkg/transformers/builder | 14.81% (**-1.85%**) | :thumbsdown: |
| github.com/xataio/pgstream/pkg/transformers/greenmask | 95.17% (**+1.65%**) | :thumbsup: |

---

<details>

<summary>Coverage by file</summary>

### Changed files (no unit tests)

| Changed File | Coverage Δ | Total | Covered | Missed | :robot: |
|--------------|------------|-------|---------|--------|---------|
| github.com/xataio/pgstream/pkg/transformers/builder/transformer_builder.go | 14.81% (**-1.85%**) | 27 (+3) | 4 | 23 (+3) | :thumbsdown: |
| github.com/xataio/pgstream/pkg/transformers/greenmask/greenmask_float_transformer.go | 97.30% (**+97.30%**) | 37 (+37) | 36 (+36) | 1 (+1) | :star2: |
| github.com/xataio/pgstream/pkg/transformers/transformer.go | 100.00% (ø) | 7 | 7 | 0 |  |

_Please note that the "Total", "Covered", and "Missed" counts above refer to ***code statements*** instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code._

### Changed unit test files

- github.com/xataio/pgstream/pkg/transformers/greenmask/greenmask_float_transformer_test.go

</details>

github-actions[bot]: ### Merging this branch changes the coverage (1 decrease, 1 increase)

| Impacted Packages | Coverage Δ | :robot: |
|-------------------|------------|---------|
| github.com/xataio/pgstream/pkg/transformers | 100.00% (ø) |  |
| github.com/xataio/pgstream/pkg/transformers/builder | 14.81% (**-1.85%**) | :thumbsdown: |
| github.com/xataio/pgstream/pkg/transformers/greenmask | 94.89% (**+1.37%**) | :thumbsup: |

---

<details>

<summary>Coverage by file</summary>

### Changed files (no unit tests)

| Changed File | Coverage Δ | Total | Covered | Missed | :robot: |
|--------------|------------|-------|---------|--------|---------|
| github.com/xataio/pgstream/pkg/transformers/builder/transformer_builder.go | 14.81% (**-1.85%**) | 27 (+3) | 4 | 23 (+3) | :thumbsdown: |
| github.com/xataio/pgstream/pkg/transformers/greenmask/greenmask_float_transformer.go | 96.55% (**+96.55%**) | 29 (+29) | 28 (+28) | 1 (+1) | :star2: |
| github.com/xataio/pgstream/pkg/transformers/transformer.go | 100.00% (ø) | 7 | 7 | 0 |  |

_Please note that the "Total", "Covered", and "Missed" counts above refer to ***code statements*** instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code._

### Changed unit test files

- github.com/xataio/pgstream/pkg/transformers/greenmask/greenmask_float_transformer_test.go

</details>


Patch:
diff --git a/pkg/transformers/builder/transformer_builder.go b/pkg/transformers/builder/transformer_builder.go
index 9ebd52a..8fb2e98 100644
--- a/pkg/transformers/builder/transformer_builder.go
+++ b/pkg/transformers/builder/transformer_builder.go
@@ -22,6 +22,8 @@ func New(cfg *transformers.Config) (transformers.Transformer, error) {
 		return greenmask.NewFirstNameTransformer(cfg.Generator, cfg.Parameters)
 	case transformers.GreenmaskInteger:
 		return greenmask.NewIntegerTransformer(cfg.Generator, cfg.Parameters)
+	case transformers.GreenmaskFloat:
+		return greenmask.NewFloatTransformer(cfg.Generator, cfg.Parameters)
 	default:
 		return nil, transformers.ErrUnsupportedTransformer
 	}
diff --git a/pkg/transformers/greenmask/greenmask_float_transformer.go b/pkg/transformers/greenmask/greenmask_float_transformer.go
new file mode 100644
index 0000000..30b05b2
--- /dev/null
+++ b/pkg/transformers/greenmask/greenmask_float_transformer.go
@@ -0,0 +1,75 @@
+// SPDX-License-Identifier: Apache-2.0
+
+package greenmask
+
+import (
+	"encoding/binary"
+	"fmt"
+	"math"
+
+	greenmasktransformers "github.com/eminano/greenmask/pkg/generators/transformers"
+	"github.com/xataio/pgstream/pkg/transformers"
+)
+
+const (
+	defaultMinFloat  = math.MaxFloat32 * -1
+	defaultMaxFloat  = math.MaxFloat32
+	defaultPrecision = 2
+)
+
+type FloatTransformer struct {
+	transformer *greenmasktransformers.RandomFloat64Transformer
+}
+
+func NewFloatTransformer(generator transformers.GeneratorType, params transformers.Parameters) (*FloatTransformer, error) {
+	minValue, err := findParameter(params, "min_value", defaultMinFloat)
+	if err != nil {
+		return nil, fmt.Errorf("greenmask_float: min_value must be a float: %w", err)
+	}
+	maxValue, err := findParameter(params, "max_value", defaultMaxFloat)
+	if err != nil {
+		return nil, fmt.Errorf("greenmask_float: max_value must be a float: %w", err)
+	}
+	precision, err := findParameter(params, "precision", defaultPrecision)
+	if err != nil {
+		return nil, fmt.Errorf("greenmask_float: precision must be an integer: %w", err)
+	}
+	limiter, err := greenmasktransformers.NewFloat64Limiter(minValue, maxValue, precision)
+	if err != nil {
+		return nil, err
+	}
+	t := greenmasktransformers.NewRandomFloat64Transformer(limiter)
+
+	if err := setGenerator(t, generator); err != nil {
+		return nil, err
+	}
+
+	return &FloatTransformer{
+		transformer: t,
+	}, nil
+}
+
+func (ft *FloatTransformer) Transform(value any) (any, error) {
+	var toTransform []byte
+	switch val := value.(type) {
+	case float32:
+		toTransform = getBytesForFloat(float64(val))
+	case float64:
+		toTransform = getBytesForFloat(val)
+	case []byte:
+		toTransform = val
+	default:
+		return nil, transformers.ErrUnsupportedValueType
+	}
+	ret, err := ft.transformer.Transform(nil, toTransform)
+	if err != nil {
+		return nil, err
+	}
+	return float64(ret), nil
+}
+
+func getBytesForFloat(f float64) []byte {
+	var buf [8]byte
+	binary.BigEndian.PutUint64(buf[:], math.Float64bits(f))
+	return buf[:]
+}
diff --git a/pkg/transformers/transformer.go b/pkg/transformers/transformer.go
index 398a3cf..efb17e6 100644
--- a/pkg/transformers/transformer.go
+++ b/pkg/transformers/transformer.go
@@ -25,6 +25,7 @@ const (
 	NeosyncFirstName   TransformerType = "neosync_firstname"
 	GreenmaskFirstName TransformerType = "greenmask_firstname"
 	GreenmaskInteger   TransformerType = "greenmask_integer"
+	GreenmaskFloat     TransformerType = "greenmask_float"
 )
 
 type GeneratorType string


Test patch:
diff --git a/pkg/transformers/greenmask/greenmask_float_transformer_test.go b/pkg/transformers/greenmask/greenmask_float_transformer_test.go
new file mode 100644
index 0000000..19ddc55
--- /dev/null
+++ b/pkg/transformers/greenmask/greenmask_float_transformer_test.go
@@ -0,0 +1,212 @@
+// SPDX-License-Identifier: Apache-2.0
+
+package greenmask
+
+import (
+	"testing"
+
+	greenmasktransformers "github.com/eminano/greenmask/pkg/generators/transformers"
+
+	"github.com/stretchr/testify/require"
+	"github.com/xataio/pgstream/pkg/transformers"
+)
+
+func Test_NewFloatTransformer(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name          string
+		params        transformers.Parameters
+		generatorType transformers.GeneratorType
+		wantErr       error
+	}{
+		{
+			name: "ok - random with parameters",
+			params: map[string]any{
+				"min_value": 1.01,
+				"max_value": 10.5,
+			},
+			generatorType: transformers.Random,
+			wantErr:       nil,
+		},
+		{
+			name: "ok - deterministic with parameters",
+			params: map[string]any{
+				"min_value": 0.0000000000000000000000000000001,
+				"max_value": 100.0,
+				"precision": 44,
+			},
+			generatorType: transformers.Deterministic,
+			wantErr:       nil,
+		},
+		{
+			name:    "ok - random with default",
+			params:  map[string]any{},
+			wantErr: nil,
+		},
+		{
+			name:          "ok - deterministic with default",
+			params:        map[string]any{},
+			generatorType: transformers.Deterministic,
+			wantErr:       nil,
+		},
+		{
+			name: "error - min_value greater than max_value",
+			params: map[string]any{
+				"min_value": 10.5,
+				"max_value": 1.5,
+				"precision": 2,
+			},
+			wantErr: greenmasktransformers.ErrWrongLimits,
+		},
+		{
+			name: "error - invalid min_value type",
+			params: map[string]any{
+				"min_value": "invalid",
+				"max_value": 10.5,
+				"precision": 2,
+			},
+			generatorType: transformers.Random,
+			wantErr:       transformers.ErrInvalidParameters,
+		},
+		{
+			name: "error - invalid max_value type",
+			params: map[string]any{
+				"min_value": 1.5,
+				"max_value": "invalid",
+				"precision": 2,
+			},
+			generatorType: transformers.Deterministic,
+			wantErr:       transformers.ErrInvalidParameters,
+		},
+		{
+			name: "error - invalid precision type",
+			params: map[string]any{
+				"min_value": 1.5,
+				"precision": "invalid",
+			},
+			generatorType: transformers.Random,
+			wantErr:       transformers.ErrInvalidParameters,
+		},
+		{
+			name: "error - invalid generator type",
+			params: map[string]any{
+				"min_value": 1.5,
+				"max_value": 10.5,
+				"precision": 2,
+			},
+			generatorType: "invalid",
+			wantErr:       transformers.ErrUnsupportedGenerator,
+		},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			transformer, err := NewFloatTransformer(tc.generatorType, tc.params)
+			require.ErrorIs(t, err, tc.wantErr)
+			if err != nil {
+				return
+			}
+			require.NotNil(t, transformer)
+		})
+	}
+}
+
+func TestFloatTransformer_Transform(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name          string
+		value         any
+		params        transformers.Parameters
+		generatorType transformers.GeneratorType
+		wantErr       error
+	}{
+		{
+			name:  "ok - random with float64",
+			value: float64(5.5),
+			params: map[string]any{
+				"min_value": 9.999999999999,
+				"max_value": 10.0,
+				"precision": 12,
+			},
+			generatorType: transformers.Random,
+			wantErr:       nil,
+		},
+		{
+			name:          "ok - deterministic with float32, with default params",
+			value:         float32(5555.5),
+			generatorType: transformers.Deterministic,
+			wantErr:       nil,
+		},
+		{
+			name:  "ok - deterministic with byte slice",
+			value: []byte{0, 0, 0, 50},
+			params: map[string]any{
+				"min_value": 1.0,
+				"max_value": 100000.0000000001,
+			},
+			generatorType: transformers.Deterministic,
+			wantErr:       nil,
+		},
+		{
+			name:  "error - invalid value type",
+			value: "invalid",
+			params: map[string]any{
+				"min_value": 1.0,
+				"max_value": 10.0,
+				"precision": 2,
+			},
+			generatorType: transformers.Random,
+			wantErr:       transformers.ErrUnsupportedValueType,
+		},
+		{
+			name:  "error - nil value",
+			value: nil,
+			params: map[string]any{
+				"min_value": 1.0,
+				"max_value": 10.0,
+				"precision": 2,
+			},
+			generatorType: transformers.Random,
+			wantErr:       transformers.ErrUnsupportedValueType,
+		},
+	}
+
+	for _, tc := range tests {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Parallel()
+			transformer, err := NewFloatTransformer(tc.generatorType, tc.params)
+			require.NoError(t, err)
+
+			got, err := transformer.Transform(tc.value)
+			require.ErrorIs(t, err, tc.wantErr)
+			if err != nil {
+				return
+			}
+			result, ok := got.(float64)
+			require.True(t, ok, "expected got to be of type float64")
+
+			// check if the result is within the specified range
+			minVal, found, err := transformers.FindParameter[float64](tc.params, "min_value")
+			require.NoError(t, err)
+			if found {
+				require.True(t, result >= minVal)
+			}
+
+			maxVal, found, err := transformers.FindParameter[float64](tc.params, "max_value")
+			require.NoError(t, err)
+			if found {
+				require.True(t, result <= maxVal)
+			}
+
+			// if deterministic, check if we get the same result again
+			if tc.generatorType == transformers.Deterministic {
+				gotAgain, err := transformer.Transform(tc.value)
+				require.NoError(t, err)
+				require.Equal(t, got, gotAgain)
+			}
+		})
+	}
+}