Bug: Cannot change the number of tokens from 512 to 256 Fatal configuration error; unable to start server.
Status: Resolved
Priority: Normal
Resolution: Not A Bug
Created: 07/Jun/20 03:17
Resolved: 16/Jun/20 05:58

Hello, I was following this issue from jira: https://issues.apache.org/jira/browse/CASSANDRA-11811?jql=text%20~%20%22CassandraDaemon.java%20Cannot%20change%22
. We are using 3.11.2 and i see this error in the log while starting the cassandra, and it fails. I read the jira and understood that mutation happens, thereby doubling the number of tokens, and hence due to mismatch

INFO [main] [2020-05-28 11:05:14] OutboundTcpConnection.java:108 - OutboundTcpConnection using coalescing strategy DISABLED
INFO [HANDSHAKE-/192.168.5.53] [2020-05-28 11:05:14] OutboundTcpConnection.java:560 - Handshaking version with /192.168.5.53
INFO [main] [2020-05-28 11:05:15] StorageService.java:707 - Loading persisted ring state
INFO [main] [2020-05-28 11:05:15] StorageService.java:825 - Starting up server gossip
INFO [main] [2020-05-28 11:05:15] TokenMetadata.java:479 - Updating topology for /192.168.5.52
INFO [main] [2020-05-28 11:05:15] TokenMetadata.java:479 - Updating topology for /192.168.5.52
Cannot change the number of tokens from 512 to 256
Fatal configuration error; unable to start server. See log for stacktrace.
ERROR [main] [2020-05-28 11:05:15] CassandraDaemon.java:708 - Fatal configuration error
org.apache.cassandra.exceptions.ConfigurationException: Cannot change the number of tokens from 512 to 256
at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:989) ~[apache-cassandra-3.11.2.jar:3.11.2]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:682) ~[apache-cassandra-3.11.2.jar:3.11.2]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:613) ~[apache-cassandra-3.11.2.jar:3.11.2]
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:379) [apache-cassandra-3.11.2.jar:3.11.2]
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602) [apache-cassandra-3.11.2.jar:3.11.2]
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) [apache-cassandra-3.11.2.jar:3.11.2]
INFO [StorageServiceShutdownHook] [2020-05-28 11:05:15] HintsService.java:220 - Paused hints dispatch
INFO [StorageServiceShutdownHook] [2020-05-28 11:05:15] Gossiper.java:1540 - Announcing shutdown
INFO [StorageServiceShutdownHook] [2020-05-28 11:05:15] StorageService.java:2292 - Node /192.168.5.52 state jump to shutdown
INFO [HANDSHAKE-/192.168.5.53] [2020-05-28 11:05:15] OutboundTcpConnection.java:560 - Handshaking version with /192.168.5.53

I would like to know 
 # what would be the root cause of this error
 # How to recover from this error. Because everytime i start the Cassandra, it is blocked due to this. 

/etc/cassandra/conf/cassandra.yaml 

contains num_tokens as 256  , auto_bootstrap is not provided, i guess by default it will be true. 

INFO [main] [2020-05-28 11:05:13] StorageService.java:618 - Cassandra version: 3.11.2
INFO [main] [2020-05-28 11:05:13] StorageService.java:619 - Thrift API version: 20.1.0
INFO [main] [2020-05-28 11:05:13] StorageService.java:620 - CQL supported versions: 3.4.4 (default: 3.4.4)
INFO [main] [2020-05-28 11:05:13] StorageService.java:622 - Native protocol supported versions: 3/v3, 4/v4, 5/v5-beta (default: 4/v4)