Project: ShardingSphere (App Definition and Development / Database)
File: Nov_23_1_Integrate_SCTL_into_RAL.en.md

+++ title = "Integrating SCTL into RAL" weight = 27 chapter = true +++ In the previous article written by Haoran Meng, the Apache ShardingSphere Committer shared the motivating reasons behind the design of DistSQL, explained its syntax system, and impressively showcased how you can use one SQL to create a sharding table. Recently, the ShardingSphere community has redesigned the SCTL grammar and the execution engine, integrating SCTL into the DistSQL syntax system. Now RAL contains the old SCTL function, making ShardingSphere's command language even more convenient for database management. Today, our community author would like to introduce the changes and elaborate on how you can use the new RAL command lines. We always pursue better user experience, and the upgrade we developed this time is just another typical example. RAL is a subtype of DistSQL. DistSQL contains three types: RDL, RQL and RAL. Resource & Rule Definition Language (RDL)to create, modify or delete resources and rules. Resource & Rule Query Language (RQL): to query and display resources and rules. Resource & Rule Administration Language (RAL): to control senior functions of resources and rules management. ShardingSphere Control Language (SCTL) is the command language of Apache ShardingSphere to execute a series of operations on enhanced features such as Hint, transaction type switch and sharding execution query. SCTL is made of the below commands | Command | Description | | -- | -- | | sctl:set transactiontype=XX | Change the transaction type ( LOCAL, XA, or BASE), e.g. sctl:set transactiontype=XA. | | sctl:show transaction_type | Query the transaction type. | | sctl:show cached_connections | Query the numbuer of physical database cached connections. | | sctl:explain SQL | Query the execution plan of the logic SQL, e.g. sctl:explain select * from t_order; | | sctl:hint set PRIMARY_ONLY=true | For the current connection only. Choose whether to hint at the primary database. | | sctl:hint set DatabaseShardingValue=yy | For the current connection only. The Hint setting only works for database sharding. Add the database sharding value yy. | | sctl:hint addDatabaseShardingValue xx=yy | For the current connection. Add database sharding value yy to the logical table xx.| | sctl:hint addTableShardingValue xx=yy | For the current connection. Add table sharding value yy to the logical table xx.| | sctl:hint clear | For the current connection only. Clear all hint setting.| | sctl:hint show status | For the current connection only. Query hint status: `primaryonly:true/false, shardingtype:databasesonly/databasestables`| | sctl:hint show table status | For the current connection only. Query hint database sharding value of the logical table.| The SCTL feature was released in ShardingSphere v3.1.0. At that time, we did not even create the DistSQL concept - now DistSQL can provide the new API with enriched features and consistent concepts. To avoid an excessively steep learning curve, we chose to integrate SCTL into

SCTL syntax is actually very easy to identify: it is marked with the special prefix "sctl:". Instead of using our Parser engine, parsing a SCTL command relies on string matching. Now that DistSQL is mature enough, it's time to delete these special codes and let the Parser engine handle them. Additionally, SCTL syntax is not real SQL. Apache ShardingSphere 5.0.0 has just been released, and DistSQL is already the best solution to manage resources and rules. This SQL created by ShardingSphere is truly SQL in practice - so why not integrate SCTL into DistSQL? Our community has discussed at length on how to handle the change. Finally, we decided to replace SCTL syntax with new RAL commands (see the table below): | Before | After | | -- | -- | | sctl:set transactiontype=XX | set variable transactiontype=XX | | sctl:show transactiontype | show variable transactiontype | | sctl:show cachedconnections | show variable cachedconnections | | sctl:explain SQL | preview SQL | | sctl:hint set PRIMARYONLY=true | set readwritesplitting hint source = [auto / write] | | sctl:hint set DatabaseShardingValue=yy | set sharding hint database_value = yy; | | sctl:hint addDatabaseShardingValue xx=yy | add sharding hint database_value xx= yy; | | sctl:hint addTableShardingValue xx=yy | add sharding hint table_value xx = yy | | sctl:hint clear | clear [hint / sharding hint / readwrite_splitting hint]| | sctl:hint show status | how [sharding / readwrite_splitting] hint status| | sctl:hint show table status | Catagorized into show sharding hint status | Now, Let's analyze these commands one by one: `show variable transaction_type` Query the current transaction type. Input command mysql> show variable transaction_type; Output ++ | TRANSACTION_TYPE | ++ | LOCAL | ++ `set variable transaction_type` Modify the current transaction type (LOCAL, XA, or BASE; case insensitive). Input command mysql> set variable transaction_type=XA; Output a. If successful, display "Query OK, 0 rows affected"; b. Execute `show variable transaction_type` again and the type is XA now. `show variable cached_connection` Query how many physical database cached connections. Input command mysql> show variable cached_connections; Output +--+ | CACHED_CONNECTIONS | +--+ | 0 | +--+ `preview SQL` Preview the actual SQL. Here, we give an example in read-write splitting scenario. ShardingSphere supports previewing any SQL commands. Input command mysql> preview select * from t_order; Output +--+-+ | datasource_name | sql | +--+-+ | readds0 | select * from torder ORDER BY orderid ASC | | readds1 | select * from torder ORDER BY orderid ASC | +--+-+ NoteThis is an Hint example in read-write splitting scenario. We configure two rules: read-write splitting and sharding. The configuration is the following: rules: !READWRITE_SPLITTING dataSourceGroups: ds_0: writeDataSourceName: writeds0 readDataSourceNames: readds0 ds_1: writeDataSourceName: writeds1 readDataSourceNames: readds1 !SHARDING tables: t_order: actualDataNodes: ds${0..1}.torder defaultDatabaseStrategy: standard: shardingColumn: user_id shardingAlgorithmName: database_inline defaultTableStrategy: none: shardingAlgorithms: database_inline: type: INLINE props: algorithm-expression: ds${userid % 2} `show readwrite_splitting hint status` For the current connection

Query hint status of readwrite_splitting. Input command mysql> show readwrite_splitting hint status; Output +--+ | source | +--+ | auto | +--+ `set readwrite_splitting hint source` For the current connection only. Set read-write splitting hint strategy (AUTO or WRITE). Supported source types includeAUTO and WRITE (case insensitive) . AUTO automated readwrite-splitting hint WRITEcompulsory hint at the master library Input command mysql> set readwrite_splitting hint source=write; Output a. If sucessful, show "Query OK, 0 rows affected"; b. Re-execute `show readwrite_splitting hint status`; show the ource is changed into Write; c. Execute `preview select * from t_order`and see the queried SQL will go to the master database. mysql> preview select * from t_order; +--+-+ | datasource_name | sql | +--+-+ | writeds0 | select * from torder ORDER BY orderid ASC | | writeds1 | select * from torder ORDER BY orderid ASC | +--+-+ `clear readwrite_splitting hint` For the current connection only. Clear the read-write splitting hint setting. Input command mysql> clear readwrite_splitting hint; Output a. If successful, show "Query OK, 0 rows affected"; b. Recover default of readwritesplitting hint; use `show readwritesplitting hint status` command to see the result. NoteHere is another sharding example for Hint. Hint algorithm is used for both database sharding and table sharding. The sharding configuration rules are shown below: rules: !SHARDING tables: torderitem: actualDataNodes: ds${0..1}.torderitem${0..1} databaseStrategy: hint: shardingAlgorithmName: database_inline tableStrategy: hint: shardingAlgorithmName: table_inline shardingAlgorithms: database_inline: type: HINT_INLINE props: algorithm-expression: ds_${Integer.valueOf(value) % 2} table_inline: type: HINT_INLINE props: algorithm-expression: torderitem_${Integer.valueOf(value) % 2} `show sharding hint status` For the current connection only. Query sharding hint status. Input command mysql> show sharding hint status; Output The initial status output is Verify the hint and input the command: preview select * from torderitem; Output No hint value now. Query is fully dependent on the hint. -`set sharding hint database_value;` For the current connection only. Set the Hint as for database sharding only, and add database value=1. Input command mysql> set sharding hint database_value = 1; Output a. If successful, show "Query OK, 0 rows affected"; b. Execute `show sharding hint status`; show `torderitem`'s `databaseshardingvalues` as 1. Update `shardingtype value` as `databasesonly`. c. Execute `preview select * from torderitem`; SQL all hinted to ds_1 *Note: According to the sharding rules of YAML configuration, when database_value is an odd number, hint at ds_1; when database_value is an even number, hint at ds_0. -`add sharding hint database_value;` For the current connection only. Add `torderitem`'s database sharding value. Input command mysql> add sharding hint databasevalue torder_item = 5; Output a. If successful, show "Query OK, 0 rows affected"; b. Execute `show sharding hint status`; Show `torderitem`'s `databaseshardingvalues` as 5; update `shardingtype value` as `databasestables`; c. Execute `preview select * from torderitem`; SQL commands are all hinted to ds_1 Enter the add command again to add an even

mysql> add sharding hint databasevalue torder_item = 10; Output a. If successful, show "Query OK, 0 rows affected"; b. Execute `show sharding hint status`; show `torderitem`'s `databaseshardingvalues` = '5,10' c. Execute `preview select * from torderitem`; SQL hint contains ds0 and ds1 ( Because the hint values include both odd and even number so it contains all target data sources) -`add sharding hint table_value;` For the current connection only. Add database sharding value for `torderitem`. Input command mysql> add sharding hint tablevalue torder_item = 0; Output a. If successful, show "Query OK, 0 rows affected"; b. Execute `show sharding hint status`; show `torderitem`'s `databaseshardingvalues` as '5,10' while `tableshardingvalues` is '0' c. Execute `preview select * from torderitem`; the Hint condition is shown in the figure below; Every database only queries `torderitem_0`: Note: According to the sharding rules of YAML configuration, when `tablevalue` is an odd number, hint `torderitem1`; when `databasevalue` is an even number, hint `torderitem0`. It's quite similar to `add sharding hint databasevalue`; you can set more than one hint values in `add sharding hint databasevalue`, to cover more shards. `clear sharding hint` For the current connection only. Clear sharding hint setting. Input command mysql> clear sharding hint; Output a. If successful, show "Query OK, 0 rows affected"; b. Clear sharding hint and recover default; use `show sharding hint status`; to see the result. The initial status is: `clear hint` It is a special command because it contains the features of `clear readwrite_splitting hint` and `clear sharding hint`. It can clear all hint values of read-write splitting and sharding. Use the command, and you will get the initial status. Set hint value and then execute the command; mysql> clear hint; Output a. If successful, show "Query OK, 0 rows affected"; b. Get readwritesplitting hint default and sharding hint default; use `show readwritesplitting hint status ;` or `show sharding hint status;` command to see the result. Note: Please remember: if you need to use DistSQL Hint, you need to enable the configuration`proxy-hint-enabled`of ShardingSphere-Proxy. For more information, please read: RAL not only contains all the SCTL functions, but also provides other useful administrational features including elastic scaling, instance ciruit-breaker, disabling read database for read-write splitting, etc. For more details about RAL, please consult the relevant documentation: That's all folks. If you have any questions or suggestions, feel free to comment on our GitHub Issues or Discussions sections. You're welcome to submit your pull request and start contributing to the open source community, too. We've also set up a Slack channel, where you can connect with other members of our community and discuss technology with us. *ShardingSphere Github:* *ShardingSphere Twitter:* *ShardingSphere Slack Channel:* *GitHub Issues* *Contributor Guide:* Jiang Longtao SphereEx Middleware Development Engineer & Apache ShardingSphere Committer. Currently, he is in charge of DistSQL and permission control development. Lan Chengxiang SphereEx Middleware Development Engineer & Apache ShardingSphere Contributor. He focuses on DisSQL design and development.