riptano/cassandra-dtest — pull request #13 by EnigmaCurry (2013-06-20 01:34:01+00:00)
Title: Replication test

This test looks at how data is replicated across a cluster via tracing. It compares those results to what the replication should be according to the docs.

This test is currently failing due to a few bugs: [CASSANDRA-5668](https://issues.apache.org/jira/browse/CASSANDRA-5668) and [CASSANDRA-5658](https://issues.apache.org/jira/browse/CASSANDRA-5658). As such you can see this debug output:

```
Test the NetworkTopologyStrategy on a 2DC 3:3 node cluster ... cluster ccm directory: /tmp/dtest-hp7sdS

replicas should be: set(['127.0.0.5', '127.0.0.4', '127.0.0.3', '127.0.0.1'])
replicas were: set(['127.0.0.4', '127.0.0.1'])
----------------------------------------
127.0.0.1	87	Parsing statement	Thrift:1
127.0.0.1	619	Peparing statement	Thrift:1
127.0.0.1	892	Determining replicas for mutation	Thrift:1
127.0.0.1	6329	Acquiring switchLock read lock	MutationStage:1
127.0.0.1	6373	Appending to commitlog	MutationStage:1
127.0.0.1	6501	Adding to test memtable	MutationStage:1
127.0.0.1	7282	Sending message to /127.0.0.4	WRITE-/127.0.0.4
127.0.0.4	100	Message received from /127.0.0.1	Thread-7
127.0.0.4	1443	Acquiring switchLock read lock	MutationStage:5
127.0.0.4	1491	Appending to commitlog	MutationStage:5
127.0.0.4	1628	Adding to test memtable	MutationStage:5
127.0.0.4	6851	Enqueuing response to /127.0.0.1	MutationStage:5
127.0.0.1	25	Message received from /127.0.0.4	Thread-14
127.0.0.4	7021	Sending message to /127.0.0.1	WRITE-/127.0.0.1
127.0.0.1	118	Processing response from /127.0.0.4	RequestResponseStage:1
----------------------------------------
FAIL
ERROR

======================================================================
ERROR: Test the NetworkTopologyStrategy on a 2DC 3:3 node cluster
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/ryan/git/datastax/cassandra-dtest/dtest.py", line 144, in tearDown
raise AssertionError('Unexpected error in %s node log: %s' % (node.name, errors))
AssertionError: Unexpected error in node1 node log: ['ERROR [WRITE-/127.0.0.3] 2013-06-19 21:27:23,403 OutboundTcpConnection.java (line 197) error writing to /127.0.0.3\n', 'ERROR [WRITE-/127.0.0.5] 2013-06-19 21:27:23,404 OutboundTcpConnection.java (line 197) error writing to /127.0.0.5\n']

======================================================================
FAIL: Test the NetworkTopologyStrategy on a 2DC 3:3 node cluster
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/ryan/git/datastax/cassandra-dtest/replication_test.py", line 198, in network_topology_test
self.assertEqual(stats['replicas'], replicas_should_be)
AssertionError: Items in the second set but not the first:
'127.0.0.5'
'127.0.0.3'
```

The test determined that data should have been written to four nodes, but the trace only shows that it wrote to two nodes, and then it shows in the log where it failed to write to those two logs. This is consistent with the above mentioned bugs, once they are fixed, this test should pass too.