Repository: ariga/atlas
Pull request #2735 (2024-05-01 14:08:01)

doc/md: v.0.22 blog post


Patch:
diff --git a/doc/md/cli-data-privacy.mdx b/doc/md/cli-data-privacy.mdx
new file mode 100644
index 00000000..6f4d970d
--- /dev/null
+++ b/doc/md/cli-data-privacy.mdx
@@ -0,0 +1,85 @@
+---
+title: Data Privacy and the CLI
+slug: /cli/data-privacy
+id: cli-data-privacy
+---
+As mentioned on the [Feature Compatability](/features) page, users can unlock advanced features by logging in with the
+Atlas CLI. This can be done by running the following command:
+
+```bash
+atlas login
+```
+
+After running this command, follow the instructions on the screen to log in. If needed, you will be prompted
+to create an account.
+
+## What data does the Atlas CLI collect?
+
+:::info Logged-in Users
+
+Notice that this document is only relevant to users who have logged in with the Atlas CLI. If you have not logged in,
+the CLI does not collect any data from you.
+
+:::
+
+As a database management tool, Atlas has access to a lot of sensitive data. We take privacy and security very seriously,
+and we want to be transparent about what data we collect and how we use it.
+
+If you want the full details in formal and legal language you can read our [Privacy Policy](https://ariga.io/legal/privacy)
+and [SaaS Agreement](https://ariga.io/legal/saas) which you sign when you create an account, but here is a summary of
+the data we collect in layman's terms.
+
+### Data we DON'T collect
+
+* **Your Data**: We don't collect any data from your databases. We don't know what data you have in your databases,
+  and we don't have access to it.
+* **Your Database Passwords**: We don't collect, transmit or store your database passwords.
+* **Your database schema or migrations**: Unless you explicitly use a command or flag that sends us your schema or
+  migrations (as described below), we don't collect, transmit or store your database schema, migrations or execution logs.
+
+### Data we DO collect
+
+* **Errors and Crashes** - We collect error and crash reports to help us improve the CLI. This includes things like the
+  stack trace of the error, the command you were running, and the version of the CLI you were using. We are very careful
+  not to collect and transmit any sensitive data in these reports.
+* **Data you choose to report** - Some Atlas commands, such as `migrate push` (which is used to upload your migration
+  directory to the Atlas Schema Registry) or flags (like running `schema inspect` with the `--web` or `-w` flag), naturally
+  involve sending us data. See below for more details on these commands and flags.
+* **Basic Telemetry** - We collect basic telemetry data to help us understand how users are using the CLI and to enforce
+  license restrictions.
+
+  This includes things like the commands you run, general information about the runtime environment (like OS version
+  or if you're running on GitHub Actions), your local hostname, the type of database you're using, and the version
+  of the  CLI you're using. We are very careful not to collect and transmit any sensitive data in these reports.
+
+### We only collect what you send us
+
+In principle, aside from basic telemetry and error reporting (with no sensitive data), we only collect data that you
+explicitly choose to send us. Consider the following list of commands and flags that send us data:
+
+* `schema inspect` when used with the `--web` or `-w` flag: This flag opens a web browser to view the schema of your
+  database. This the schema of your database.
+* `migrate push`: This command uploads your migration directory to the Atlas Schema Registry. This includes your schema
+  and migrations.
+* `migrate apply` when used with a directory that is stored in the schema registry (e.g the url has a `atlas://` prefix)
+  AND the `--env` flag is used: This flag applies a migration from the Atlas Schema Registry to your database and uploads
+  the execution logs to Atlas Cloud, updating the status of the target database in your account. We also store the
+  hostname of the target database to help you identify which database the logs belong to.
+* `schema apply` when used with the interactive `Lint and Edit` mode: This mode allows you to edit your schema in a
+  visual editor and then apply the changes to your database. This sends your schema to Atlas Cloud for processing.
+* `migrate lint` when used with the `--web` or `-w` flag: This flag opens a web browser to view the linting results of
+  your migrations. This sends your migrations to Atlas Cloud for processing.
+
+## How do we use the data we collect?
+
+We use the data we collect to improve the CLI and provide better support to our users. This includes things like:
+
+* Fixing bugs and crashes
+* Improving the performance of the CLI
+* Understanding how users are using the CLI
+* Providing better experience to users
+
+We don't sell your data to third parties, and use it strictly in compliance with our [Privacy Policy](https://ariga.io/legal/privacy)
+as well as applicable laws and regulations.  We take data privacy and security very seriously and hold ourselves accountable
+to the highest standards.  To ensure this commitment is upheld we perform an annual SOC 2 audit which you can read more
+[about here](https://ariga.io/announcing-soc2-compliance-for-atlas-cloud/).
\ No newline at end of file
diff --git a/doc/website/blog/2024-05-01-atlas-v-0-22.mdx b/doc/website/blog/2024-05-01-atlas-v-0-22.mdx
new file mode 100644
index 00000000..9ea28384
--- /dev/null
+++ b/doc/website/blog/2024-05-01-atlas-v-0-22.mdx
@@ -0,0 +1,198 @@
+---
+title: "Announcing v0.22: Rename Detection, Table Locking Checks, and more"
+authors: hilakashai
+tags: [release, postgresql, sql-server, ]
+---
+
+Hi everyone,
+
+It's been a few weeks since our [last release](https://github.com/ariga/atlas/releases/tag/v0.21.0), and we're happy to be back with a version packed with brand new and exciting
+features. Here's what's inside:
+
+* [**`RENAME` Detection**](#rename-detection) - This version includes a `RENAME` detector that identifies ambiguous situations of potential resource renames  and interactively asks the user for feedback before generating the changes.
+* [**PostgreSQL Features**](#postgresql-features)
+    * [**UNIQUE and EXCLUDE**](#unique-and-exclude-constraints) - Unique constraints and exclusion constraints were added.
+    * [**Composite Types**](#composite-types) - Added support for composite types, which are user-defined types that represent the structure of a row.
+    * [**Table lock checks**](#table-locking-checks) - Eight new checks that review migration plans and alert in cases of a potential table locks in different modes.
+* [**SQL Server Sequence Support**](#sql-server-sequence-support) -  Atlas now supports managing sequences in SQL Server.
+
+Let's dive in!
+
+## `RENAME` Detection
+
+One of the first things we were asked when we introduced Atlas's declarative approach to schema management was,
+“How are you going to deal with renames?” While column and table renames are a fairly rare event in the lifecycle of a project,
+the question arises from the fact that it's impossible to completely disambiguate between `RENAME` and `DROP` and `ADD` operations.
+While the end schema will be the same in both cases, the actual impact of an undesired `DROP` operation can be disastrous.
+
+To avoid this, Atlas now detects potential `RENAME` scenarios during the migration planning phase and prompts the user
+about their intent.
+
+Let's see this in action.
+
+Assume we have a `users` table with the column `first_name`, which we changed to `name`.
+
+After running the [`atlas migrate diff`](/cli-reference#atlas-migrate-diff) command to generate a migration, we will see the following:
+
+```bash
+? Did you rename "users" column from "first_name" to "name":
+▸ Yes
+  No
+```
+If this was our intention, we will click "Yes" and the SQL statement will be a `RENAME` statement.
+
+If we click "No", the SQL statement will drop the `first_name` column and create the `name` column instead.
+
+## PostgreSQL Features
+
+### Unique and Exclude Constraints
+
+Now, Atlas supports declaring unique and exclude constraints in your schema.
+
+For example, if we were to add a unique constraint on a
+`name` column, it would look similar to:
+
+```hcl title="schema.hcl"
+# Columns only.
+unique "name" {
+ columns = [column.name]
+}
+```
+Read more about unique constraints in Atlas [here](/atlas-schema/hcl#unique-constraints).
+
+[Exclude constraints](https://www.postgresql.org/docs/current/ddl-constraints.html#DDL-CONSTRAINTS-EXCLUSION) ensure that if any
+two rows are compared using a specified operator, at least one of the specified conditions must hold true. This means
+that the constraint ensures that no two rows satisfy the specified operator at the same time.
+
+```hcl title="schema.hcl"
+exclude "excl_speaker_during" {
+  type = GIST
+  on {
+    column = column.speaker
+    op     = "="
+  }
+  on {
+    column = column.during
+    op     = "&&"
+  }
+}
+```
+
+
+### Composite Types
+[Composite Types](https://www.postgresql.org/docs/current/rowtypes.html#ROWTYPES) are user-defined data types that represent a structure of a row or record.
+Once defined, composite types can be used to declare columns in tables or used in functions and stored procedures.
+
+For example, let's say we have a `users` table where each user has an `address`. We can create a composite type `address`
+and add it as a column to the `users` table:
+
+```hcl title="schema.hcl"
+composite "address" {
+  schema = schema.public
+  field "street" {
+    type = text
+  }
+  field "city" {
+    type = text
+  }
+}
+
+table "users" {
+  schema = schema.public
+  column "address" {
+    type = composite.address
+  }
+}
+
+schema "public" {
+  comment = "standard public schema"
+}
+```
+Learn more about composite types [here](/atlas-schema/hcl#composite-type).
+
+### Table Locking Checks
+
+One of the common ways in which schema migration cause production outages is when a schema change requires the database to acquire a lock on a table,
+immediately causing read or write operations to fail. If you are dealing with small tables, these locks might be acquired for a
+short time which will not be noticeable. However, if you are managing a large and busy database, these situations can lead to a full-blown system outage.
+
+Many developers are not aware of these pitfalls only to discover them in the middle of a crisis, which is made even worse by the fact
+that once they happen, there's nothing you can do except quietly wait for the migration to complete.
+
+Teams looking to improve the reliability and stability of their systems, reach out to automation
+to prevent human errors like these. Atlas's [automatic analysis](/versioned/lint) capabilities can be utilized to detect such risky
+changes during the CI phase of the software development lifecycle.
+
+In this version, we have added eight new [analyzers](lint/analyzers#analyzers) to our PostgreSQL integration that check for cases where planned migrations
+can lead to locking a table. Here's a short rundown of these analyzers and what they detect:
+
+* [PG104](/lint/analyzers#PG104) - Adding a `PRIMARY KEY` constraint (with its index) acquires an `ACCESS EXCLUSIVE` lock on the table,
+blocking all access during the operation.
+* [PG105](lint/analyzers#PG105) - Adding a `UNIQUE` constraint (with its index) acquires an `ACCESS EXCLUSIVE` lock on the table,
+blocking all access during the operation.
+* [PG301](lint/analyzers#PG301) - A change to the column type that requires rewriting the table (and potentially its indexes) on
+disk.
+* [PG302](lint/analyzers#PG302) - Adding a column with a volatile DEFAULT value requires a rewrite of the table.
+* [PG303](lint/analyzers#PG303) - Modifying a column from NULL to NOT NULL requires a full table scan.
+    * If the table has a CHECK constraint that ensures NULL cannot exist, such as CHECK (c > 10 AND c IS NOT NULL), the table scan is skipped, and therefore this check is not reported.
+* [PG304](lint/analyzers#PG304) - Adding a PRIMARY KEY on a nullable column implicitly sets them to NOT NULL, resulting in a full table scan unless there is a CHECK constraint that ensures NULL cannot exist.
+* [PG305](lint/analyzers#PG305) - Adding a CHECK constraint without the NOT VALID clause requires scanning the table to verify that all rows satisfy the constraint.
+* [PG306](lint/analyzers#PG306) - Adding a FOREIGN KEY constraint without the NOT VALID clause requires a full table scan to verify that all rows satisfy the constraint.
+
+View a full list of all the checks [here](/lint/analyzers#checks).
+
+
+## SQL Server Sequence Support
+
+In SQL Server, [sequences](https://learn.microsoft.com/en-us/sql/relational-databases/sequence-numbers/sequence-numbers?view=sql-server-ver16) are
+objects that generate a sequence of numeric values according to specific properties.Sequences are often used to generate
+unique identifiers for rows in a table.
+
+Atlas supports the different types of sequences. For example, a simple sequence with default values can be declared like so:
+
+```hcl
+sequence "s1" {
+  schema = schema.dbo
+}
+```
+
+We can also create a sequence with a custom configuration.
+
+```hcl
+sequence "s2" {
+  schema    = schema.dbo
+  type      = int
+  start     = 1001
+  increment = 1
+  min_value = 1001
+  max_value = 9999
+  cycle     = true
+}
+```
+In the example above, we have a sequence that starts at 1001, and is incremented by 1 until it reaches the maximum value of 9999. Once it reaches
+its maximum value, it will start over because `cycle` is set to true.
+
+Another option is to create a sequence with an alias type. For example, if we were to create a sequence for
+Social Security Numbers, we would do the following:
+
+```hcl
+sequence "s3" {
+  schema    = schema.dbo
+  type      = type_alias.ssn
+  start     = 111111111
+  increment = 1
+  min_value = 111111111
+}
+type_alias "ssn" {
+  schema = schema.dbo
+  type   = dec(9, 0)
+  null   = false
+}
+```
+
+Read the [docs](/atlas-schema/hcl#sequence) for more information about sequences.
+
+## Wrapping Up
+
+That's all for this release! We hope you try out (and enjoy) all of these new features and find them useful.
+As always, we would love to hear your feedback and suggestions on our [Discord server](https://discord.com/invite/zZ6sWVg6NT).
diff --git a/doc/website/docusaurus.config.js b/doc/website/docusaurus.config.js
index 63d420e8..46107418 100644
--- a/doc/website/docusaurus.config.js
+++ b/doc/website/docusaurus.config.js
@@ -149,7 +149,7 @@ module.exports = {
     },
     announcementBar: {
       id: 'announcementBar-12', // Increment on change
-      content: `<strong>Atlas v0.21: Introducing Atlas Migrate Down </strong><a class="cta" href="https://atlasgo.io/blog/2024/04/01/migrate-down">Read it here</a>`,
+      content: `<strong>Atlas v0.22: Rename Detection, Table Locking Checks, and more. </strong><a class="cta" href="https://atlasgo.io/blog/2024/05/01/atlas-v-0-22">Read it here</a>`,
       isCloseable: true,
     },
   },
diff --git a/doc/website/sidebars.js b/doc/website/sidebars.js
index 5521d74e..f9ec5cd7 100644
--- a/doc/website/sidebars.js
+++ b/doc/website/sidebars.js
@@ -152,6 +152,11 @@ module.exports = {
                     type: 'doc',
                     id: 'community-edition'
                 },
+                {
+                    type: 'doc',
+                    id: 'cli-data-privacy',
+                    label: 'Data Privacy'
+                }
             ]
         },
         {
diff --git a/sql/internal/sqlx/plan.go b/sql/internal/sqlx/plan.go
index 955f3702..3c749f32 100644
--- a/sql/internal/sqlx/plan.go
+++ b/sql/internal/sqlx/plan.go
@@ -382,8 +382,16 @@ func byKeys[T any](m map[string]T) []struct {
 	return vs
 }
 
+// SortOptions allows drivers to customize the behavior of the SortChanges function.
+type SortOptions struct {
+	// FuncDepT reports if a function depends on the given table.
+	FuncDepT func(*schema.Func, *schema.Table) bool
+	// FuncDepO reports if a function depends on the given object.
+	FuncDepO func(*schema.Func, schema.Object) bool
+}
+
 // SortChanges is a helper function to sort to level changes based on their priority.
-func SortChanges(changes []schema.Change) []schema.Change {
+func SortChanges(changes []schema.Change, opts *SortOptions) []schema.Change {
 	var views, drop, other []schema.Change
 	for _, c := range changes {
 		switch c.(type) {
@@ -404,7 +412,7 @@ func SortChanges(changes []schema.Change) []schema.Change {
 	edges := make(map[schema.Change][]schema.Change)
 	for _, c := range changes {
 		for _, c2 := range changes {
-			if c != c2 && dependsOn(c, c2) {
+			if c != c2 && dependsOn(c, c2, V(opts)) {
 				edges[c] = append(edges[c], c2)
 			}
 		}
@@ -444,7 +452,7 @@ type Depender interface {
 }
 
 // dependsOn reports if the given change depends on the other change.
-func dependsOn(c1, c2 schema.Change) bool {
+func dependsOn(c1, c2 schema.Change, opts SortOptions) bool {
 	if dependOnOf(c1, c2) {
 		return true
 	}
@@ -586,6 +594,10 @@ func dependsOn(c1, c2 schema.Change) bool {
 			if funcDep(c1.F, c2.To) {
 				return true // Relies on the new definition.
 			}
+		case *schema.AddTable:
+			if opts.FuncDepT != nil && opts.FuncDepT(c1.F, c2.T) {
+				return true
+			}
 		case *schema.AddObject:
 			t, ok := c2.O.(schema.Type)
 			if ok && (c1.F.Ret == t || slices.ContainsFunc(c1.F.Args, func(f *schema.FuncArg) bool {
diff --git a/sql/mysql/migrate.go b/sql/mysql/migrate.go
index 4341f66b..c627d7ee 100644
--- a/sql/mysql/migrate.go
+++ b/sql/mysql/migrate.go
@@ -85,7 +85,7 @@ func (s *state) plan(changes []schema.Change) error {
 	if err != nil {
 		return err
 	}
-	for _, c := range sqlx.SortChanges(planned) {
+	for _, c := range sqlx.SortChanges(planned, nil) {
 		switch c := c.(type) {
 		case *schema.AddTable:
 			err = s.addTable(c)
diff --git a/sql/postgres/driver_oss.go b/sql/postgres/driver_oss.go
index cef91b67..0f7572a6 100644
--- a/sql/postgres/driver_oss.go
+++ b/sql/postgres/driver_oss.go
@@ -356,6 +356,10 @@ func convertExclude(schemahcl.Resource, *schema.Table) error {
 	return nil // unimplemented.
 }
 
+func (*state) sortChanges(changes []schema.Change) []schema.Change {
+	return sqlx.SortChanges(changes, nil)
+}
+
 func detachCycles(changes []schema.Change) ([]schema.Change, error) {
 	return sqlx.DetachCycles(changes)
 }
diff --git a/sql/postgres/migrate.go b/sql/postgres/migrate.go
index 939ef799..e5f87de6 100644
--- a/sql/postgres/migrate.go
+++ b/sql/postgres/migrate.go
@@ -80,7 +80,7 @@ func (s *state) plan(changes []schema.Change) error {
 	if planned, err = detachCycles(planned); err != nil {
 		return err
 	}
-	for _, c := range sqlx.SortChanges(planned) {
+	for _, c := range s.sortChanges(planned) {
 		switch c := c.(type) {
 		case *schema.AddTable:
 			err = s.addTable(c)


Test patch:
diff --git a/sql/internal/sqlx/plan_test.go b/sql/internal/sqlx/plan_test.go
index 297e4b06..274b6a45 100644
--- a/sql/internal/sqlx/plan_test.go
+++ b/sql/internal/sqlx/plan_test.go
@@ -164,7 +164,7 @@ func TestBadTypeComparison(t *testing.T) {
 	SortChanges([]schema.Change{
 		&schema.DropObject{O: o1},
 		&schema.DropTable{T: schema.NewTable("t1").AddColumns(schema.NewColumn("c1").SetType(o1))},
-	})
+	}, nil)
 }
 
 func TestSortChanges(t *testing.T) {
@@ -186,7 +186,7 @@ func TestSortChanges(t *testing.T) {
 		&schema.AddTable{T: t1},
 		&schema.AddFunc{F: f1},
 	}
-	planned := SortChanges(changes)
+	planned := SortChanges(changes, nil)
 	require.Equal(t, []schema.Change{changes[2], changes[0], changes[3], changes[1]}, planned)
 
 	changes = []schema.Change{
@@ -194,14 +194,14 @@ func TestSortChanges(t *testing.T) {
 		&schema.DropFunc{F: f1},
 		&schema.DropTrigger{T: tr1},
 	}
-	planned = SortChanges(changes)
+	planned = SortChanges(changes, nil)
 	require.Equal(t, []schema.Change{changes[2], changes[1], changes[0]}, planned)
 
 	// No changes.
 	planned = SortChanges([]schema.Change{
 		&schema.DropFunc{F: f1},
 		&schema.DropTable{T: t1},
-	})
+	}, nil)
 	require.Equal(t, []schema.Change{&schema.DropFunc{F: f1}, &schema.DropTable{T: t1}}, planned)
 
 	// The table must be dropped before the enum type if one of its triggers depends on the enum type.
@@ -210,7 +210,7 @@ func TestSortChanges(t *testing.T) {
 		&schema.DropObject{O: e1},
 		&schema.DropTable{T: t1},
 	}
-	planned = SortChanges(changes)
+	planned = SortChanges(changes, nil)
 	require.Equal(t, []schema.Change{changes[1], changes[0]}, planned)
 
 	// The table must be dropped before the function if one of its triggers depends on the function.
@@ -218,7 +218,7 @@ func TestSortChanges(t *testing.T) {
 	planned = SortChanges([]schema.Change{
 		&schema.DropFunc{F: f1},
 		&schema.DropTable{T: t1},
-	})
+	}, nil)
 	require.Equal(t, []schema.Change{&schema.DropTable{T: t1}, &schema.DropFunc{F: f1}}, planned)
 
 	// Ignore functions that do not reside on the same schema.
@@ -227,7 +227,7 @@ func TestSortChanges(t *testing.T) {
 	planned = SortChanges([]schema.Change{
 		&schema.AddFunc{F: f1},
 		&schema.DropFunc{F: f2},
-	})
+	}, nil)
 	require.Equal(t, []schema.Change{&schema.AddFunc{F: f1}, &schema.DropFunc{F: f2}}, planned)
 
 	// Order functions that reside on the same schema.
@@ -235,7 +235,7 @@ func TestSortChanges(t *testing.T) {
 	planned = SortChanges([]schema.Change{
 		&schema.AddFunc{F: f1},
 		&schema.DropFunc{F: f2},
-	})
+	}, nil)
 	require.Equal(t, []schema.Change{&schema.DropFunc{F: f2}, &schema.AddFunc{F: f1}}, planned)
 }