av9ash/GitBugs huggingface dataset:
Bug: 15583 - Tooling testing foundation
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Jul/20 10:02
Resolved: 17/Jul/20 07:48

Tooling testing is not as thorough as we'd like it to be. The purpose of this ticket, under the umbrella of CASSANDRA-15583, is to provide the foundations to make it easy for devs, migrate the few existing tests as a POC and put that forward as a proposal to build on top of.
Bug: test_cleanup - bootstrap_test.TestBootstrap
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 07/Aug/20 00:19
Resolved: 28/Aug/20 09:41

The test is seen failing in CircleCI [here | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/295/workflows/168d88ab-f55f-4560-a23e-8243aff7b1bd/jobs/1777]
Bug: 15583 - Ensure tooling surface area coverage
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Jul/20 07:55
Resolved: 22/Jul/20 14:54

Given CASSANDRA-15942 POC seems ok so far. Now we'll extend that to other tools and scenarios in a width first approach to make sure enough surface area is covered. This should suffice to confirm our framework should enable devs to test any tools when needed.

Tools: https://cassandra.apache.org/doc/latest/tools/index.html

||Tool||Easy UT possible||UT Sample||Easy dtest possible||Has dtest||Comments||
|Nodetool|(/) Added/Enhanced|(/) Added/enhanced|(/)|(/)|Use UT if possible. Otherwise dtests if you need a full multinode env|
|Cqlsh|(/) Added|(/) Added|(/)|(/)|Easy Cqlsh UT is now possible and an example is included|
|Cassandra-stress|(/) Added and existing|(/) Added|(/)|(x)|
|debug-cql|(/)|(x)|(/)|(x)|
|fqltool|(/) Existing|(/) Existing|(/)|(/)|
|auditlogviewer|(/) Existing|(/) Existing|(/)|(/)|
|*Sstable utilities*||||||
|sstabledump|(/) enhanced|(/) enhanced|(/)|(/)||
|sstableexpiredblockers|(/) enhanced|(/) enhanced|(/)|(/)||
|sstablelevelreset|(/) enhanced|(/) enhanced|(/)|(/)||
|sstableloader|(/) enhanced|(/) enhanced|(/)|(/)||
|sstablemetadata|(/) enhanced|(/) enhanced|(/)|(!)|Ran in dtests, no dedicated test|
|sstableofflinerelevel|(/) enhanced|(/) enhanced|(/)|(/)||
|sstablerepairedset|(/) enhanced|(/) enhanced|(/)|(!)|Ran in dtests, no dedicated test|
|sstablescrub|(/) enhanced|(/) enhanced|(/)|(/)||
|sstablesplit|(/) enhanced|(/) enhanced|(/)|(/)||
|sstableupgrade|(/) enhanced|(/) enhanced|(/)|(/)||
|sstableutil|(/) enhanced|(/) enhanced|(/)|(/)||
|sstableverify|(/) enhanced|(/) enhanced|(/)|(/)||
Bug: Support fetching all user tables to compare in Cassandra-diff
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 16/Jul/20 23:15
Resolved: 27/Jul/20 11:16

The spark diff job may fail to launch with kernel error "E2BIG: Argument list too long", when passing a large list of keyspace table list to compare. 

Proposing a mode to fetch all user tables from the clusters to be compared. When the mode is on, the spark job ignores the parameter "keyspace_tables".
Bug: Avoid NPE in getCompactionInfo
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 20/Aug/20 06:50
Resolved: 24/Aug/20 11:16

We currently initialize {{sstables}} after calling {{beginCompaction(this)}} in the {{CompactionIterator}} constructor which creates a window where we can get NPE creating the {{CompactionInfo}} for cancelling compactions
Bug: Consolidate dead nodes check in force repair
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Sep/20 19:32
Resolved: 09/Oct/20 20:52

The check for dead nodes during force repair is duplicated in the normal and incremental repair. We could consolidate those 2 checks to make the code more dry. 
The check should throw a more meaningful error message to indicate that all neighbor nodes are down, instead of "java.lang.IllegalArgumentException: Endpoints can not be empty"
Bug: TestBootstrap::test_cleanup failing on unexpected number of SSTables
Status: Resolved
Priority: Normal
Resolution: Later
Created: 29/Jul/20 15:17
Resolved: 27/Oct/20 19:50

This failure has now occurred in a number of places on trunk (4.0), including both Java 8 and 11 dtest runs. Nominally, there appear to be more SSTables after cleanup than the test is expecting.

{noformat}
if len(sstables) > basecount + jobs:
    logger.debug("Current count is {}, basecount was {}".format(len(sstables), basecount))
    failed.set()
{noformat}

Examples:

https://app.circleci.com/pipelines/github/maedhroz/cassandra/92/workflows/c59be4f8-329e-4d76-9c59-d49c38e58dd2/jobs/448

https://app.circleci.com/pipelines/github/jolynch/cassandra/20/workflows/9d6c3b86-6207-4ead-aa4b-79022fc84182/jobs/893
Bug: test_resume_stopped_build - materialized_views_test.TestMaterializedViews
Status: Resolved
Priority: Normal
Resolution: Cannot Reproduce
Created: 07/Aug/20 00:05
Resolved: 25/Sep/20 16:52

Failing in CircleCI [here | https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/295/workflows/168d88ab-f55f-4560-a23e-8243aff7b1bd/jobs/1774]
Bug: Avoid "involved in several compactions" exception 
Status: Resolved
Priority: Normal
Resolution: Duplicate
Created: 07/Sep/20 08:06
Resolved: 07/Sep/20 09:23

When aborting compactions we check if the sstable is involved in other anticompactions by grabbing all compactions for a given sstable from ActiveCompactions, it asserts that an sstable is only involved in a single "compaction" here, which is wrong - it can be in several, for example a validation + a regular compaction.
Bug: Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 13/Aug/20 22:27
Resolved: 19/Oct/20 19:21

Some compact storage tables, specifically those where the user has defined both at least one clustering and the value column, can be safely handled in 4.0 because besides the DENSE flag they are not materially different post 3.0 and there is no visible change to the user facing schema after dropping compact storage. We can detect this case and allow these tables to silently drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE tables that don’t meet the criteria.
Bug: Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 08/Sep/20 21:00
Resolved: 03/Nov/20 13:47

Build: https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681

{code}
java.lang.ClassCastException: org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to org.apache.cassandra.transport.messages.ResultMessage$SchemaChange
	at org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916)
	at org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94)
{code}
Bug: org.apache.cassandra.repair.RepairJobTest testOptimizedCreateStandardSyncTasks
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Jul/20 23:33
Resolved: 13/Nov/20 11:04

Build: https://ci-cassandra.apache.org/job/Cassandra-trunk-test/lastCompletedBuild/testReport/junit/org.apache.cassandra.repair/RepairJobTest/testOptimizedCreateStandardSyncTasks/

Expecting:
 <[#<TreeRange (00000002,00000003] depth=0>,
   #<TreeRange (00000000,00000001] depth=0>]>
to contain only:
 <[(00000000,00000001]]>
but the following elements were unexpected:
 <[#<TreeRange (00000002,00000003] depth=0>]>

This failed 3 times in a row on Jenkins
Bug: Improve dtest runtime
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 27/Aug/20 19:21
Resolved: 16/Dec/20 17:11

A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a [30% increase in run time|https://www.mail-archive.com/dev@cassandra.apache.org/msg15606.html]. While that change was accepted, we wanted to spin out a ticket to optimize dtests in an attempt to gain back some of that runtime.

At this time we don't have concrete improvements in mind, so the first order of this ticket will be to analyze the state of things currently, and try to ascertain some valuable optimizations. Once the problems are understood, we will break down subtasks to divide the work.

Some areas to consider:
* cluster reuse
* C* startup optimizations
* Tests that should be ported to in-JVM dtest or even unit tests
Bug: unable to start the cassandra on windows 10 machine
Status: Triage Needed
Priority: Normal
Created: 15/Jul/20 22:30

Os - Windows 10

Cassandra -  apache-cassandra-3.11.4

java version - java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)

python version - Python 3.8.4

I am getting below error when starting cassandra -f

INFO [main] 2020-07-15 17:20:54,566 SigarLibrary.java:44 - Initializing SIGAR library
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000010014ed4, pid=13704, tid=0x00000000000025f4
#
# JRE version: Java(TM) SE Runtime Environment (8.0_261-b12) (build 1.8.0_261-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.261-b12 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C [sigar-amd64-winnt.dll+0x14ed4]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\cass311\apache-cassandra-3.11.4\bin\hs_err_pid13704.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Bug: max_compaction_flush_memory_in_mb is interpreted as bytes
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 24/Aug/20 09:39
Resolved: 25/Aug/20 18:24

In CASSANDRA-12662, [~scottcarey] [reported|https://issues.apache.org/jira/browse/CASSANDRA-12662?focusedCommentId=17070055&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17070055] that the {{max_compaction_flush_memory_in_mb}} setting gets incorrectly interpreted in bytes rather than megabytes as its name implies.

{quote}
1.  the setting 'max_compaction_flush_memory_in_mb' is a misnomer, it is actually memory in BYTES.  If you take it at face value, and set it to say, '512' thinking that means 512MB,  you will produce a million temp files rather quickly in a large compaction, which will exhaust even large values of max_map_count rapidly, and get the OOM: Map Error issue above and possibly have a very difficult situation to get a cluster back into a place where nodes aren't crashing while initilaizing or soon after.  This issue is minor if you know about it in advance and set the value IN BYTES.
{quote}
Bug: Loss of functionality around null clustering when dropping compact storage
Status: Open
Priority: Normal
Created: 21/Aug/20 16:31

For backward compatibility reasons[1], it is allowed to insert rows where some of the clustering columns are {{null}} for compact tables. That support is a tad limited/inconsistent[2] but essentially you can do:
{noformat}
cqlsh:ks> CREATE TABLE t (k int, c1 int, c2 int, v int, PRIMARY KEY (k, c1, c2)) WITH COMPACT STORAGE;
cqlsh:ks> INSERT INTO t(k, c1, v) VALUES (1, 1, 1);
cqlsh:ks> SELECT * FROM t;

 k | c1 | c2   | v
---+----+------+---
 1 |  1 | null | 1

(1 rows)
cqlsh:ks> UPDATE t SET v = 2 WHERE k = 1 AND c1 = 1;
cqlsh:ks> SELECT * FROM t;

 k | c1 | c2   | v
---+----+------+---
 1 |  1 | null | 2

(1 rows)
{noformat}
This is not allowed on non-compact tables however:
{noformat}
cqlsh:ks> CREATE TABLE t2 (k int, c1 int, c2 int, v int, PRIMARY KEY (k, c1, c2));
cqlsh:ks> INSERT INTO t2(k, c1, v) VALUES (1, 1, 1);
InvalidRequest: Error from server: code=2200 [Invalid query] message="Some clustering keys are missing: c2"
cqlsh:ks> UPDATE t2 SET v = 2 WHERE k = 1 AND c1 = 1;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Some clustering keys are missing: c2"
{noformat}
Which means that a user with a compact table that rely on this will not be able to use {{DROP COMPACT STORAGE}}.

Which is a problem for the 4.0 upgrade story. Problem to which we need an answer.

 
----
[1]: the underlying {{CompositeType}} used by such tables allows to provide only a prefix of components, so thrift users could have used such functionality. We thus had to support it in CQL, or those users wouldn't have been able to upgrade to CQL easily.

[2]: building on the example above, the value for {{c2}} is essentially {{null}}, yet none of the following is currently allowed:
{noformat}
cqlsh:ks> INSERT INTO t(k, c1, c2, v) VALUES (1, 1, null, 1);
InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid null value in condition for column c2"
cqlsh:ks> UPDATE t SET v = 2 WHERE k = 1 AND c1 = 1 AND c2 = null;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid null value in condition for column c2"
cqlsh:ks> SELECT * FROM c WHERE k = 1 AND c1 = 1 AND c2 = null;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid null value in condition for column c2"
{noformat}
Not only is that unintuitive/inconsistent, but the {{SELECT}} one means there is no way to select only the row. You can skip specifying {{c2}} in the {{SELECT}}, but this become a slice selection essentially, as shown below:
{noformat}
cqlsh:ks> INSERT INTO ct(k, c1, c2, v) VALUES (1, 1, 1, 1);
cqlsh:ks> SELECT * FROM ct WHERE k = 1 AND c1 = 1;

 k | c1 | c2   | v
---+----+------+---
 1 |  1 | null | 1
 1 |  1 |    1 | 1

(2 rows)
{noformat}
Bug: sstableupgrade unit test hardening
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 04/Aug/20 08:54
Resolved: 04/Nov/20 18:02

During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage for this tool is minimal. There is a unit test to enhance upon under {{test/unit/org/apache/cassandra/tools}}.
Bug: In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 30/Sep/20 23:13
Resolved: 22/Oct/20 20:26

Some convenience improvements to in-JVM dtest that are useful across versions that I needed while working on CASSANDRA-16144

* Add support for changing schema with stopped nodes.

* Make it simpler to modify nested configuration items by specifying Yaml fragments
Bug: Record the expiration time for hints files to avoid loading expired ones
Status: Triage Needed
Priority: Normal
Created: 30/Sep/20 00:05

The expiration time of a hints file is considered to be the latest expiration time among all the hints in the file. If the current time exceeds the file expiration time, the file can be safely deleted. 

The expiration time can be determined when finishing writing to the hints file. 

The tricky part is that each hints file keeps the metadata at the header of the file, but the expiration time is only known at the end. So we may want to save the metadata in a companion file of the hints. This approach is also future-proof, in that case that we want to add more metadata.
Bug: Distinguish partition errors published by Cassandra-Diff between source and target
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 20/Aug/20 22:13
Resolved: 03/Sep/20 07:42

The partition errors found during diff are persisted without the error origin information. 

Therefore, I am proposing to add an error origin indicator, (e.g. 0 for source and 1 for target) when persisting the partition error details.
Bug: Allow optional query retry in cassandra-diff
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 14/Sep/20 19:33
Resolved: 21/Sep/20 08:03

The diff job fails when seeing the first query timeout/unavailable. 

We can add an optional retry policy to add the spark workers retry the failed query for a maximum amount of duration (i.e. retry timeout). 

When the retry policy is set, the job only fails after the retry timeout exceeds.
Bug: Enable Moving Tokens on Nodes with Multiple Tokens
Status: Triage Needed
Priority: Normal
Created: 23/Sep/20 14:31

Currently it’s only possible to move tokens on nodes with single tokens, but the protocol introduced on CASSANDRA-16139 makes this a possibility on nodes with multiple tokens.

This may be useful to perform ring rebalancing operations. A pseudo-code for the move token operation is available [here|https://gist.github.com/pauloricardomg/1930c8cf645aa63387a57bb57f79a0f7#file-move-py].

This ticket is to enable this functionality to users via ringtool (CASSANDRA-16140).
Bug: Enable Adding and Removing Tokens to a Node (Dynamic Virtual Nodes)
Status: Triage Needed
Priority: Normal
Created: 23/Sep/20 14:28

The new ring membership protocol added on CASSANDRA-16139 allows adding and removing tokens to an existing cluster node.

This allows operators to increase/reduce the number of virtual nodes per node on a live cluster. Furthermore this also enables Incremental Bootstrap (CASSANDRA-8494).

This ticket is to enable this functionality to users via ringtool (CASSANDRA-16140).

The pseudo-code for each operation is available below:
* [add token|https://gist.github.com/pauloricardomg/1930c8cf645aa63387a57bb57f79a0f7#file-bootstrap-py] (essentially similar to bootstrap)
* [remove token|https://gist.github.com/pauloricardomg/1930c8cf645aa63387a57bb57f79a0f7#file-remove_token-py]
Bug: Use materialized view, CPU usage over 100%
Status: Triage Needed
Priority: Normal
Created: 12/Sep/20 09:02

env info:

os: CentOS Linux release 7.4.1708

cassandra: apache-cassandra-3.11.8

jdk:1.8.0_261

 

I used materialized view,but the cpu use over 100 when not cql client request, 

My analysis process is as follows:

1. top

 find pid is:25640

2. top -pH 25640

!image-2020-09-12-16-52-46-581.png!

3. printf "%x\n" 26065

convert  threadid 26065 ,hex value is :65d1

4.jstack -l 26065 > jstack_25640.txt

find 65d1 ind jstask_25640.txt

!image-2020-09-12-16-55-49-813.png!

 

5.find in source code on org.apache.cassandra.db.Keyspace.applyInternal,line 545

I guess used cpu over 100% caused by a loop call Keyspace.applyInternal.

 

Everybody have any suggessted?

The jstack_25640.txt file on Attachment.
Bug: Blog / Docs - Create Cassandra on Kubernetes Content 
Status: Triage Needed
Priority: Normal
Created: 10/Sep/20 20:28

The Cassandra Kubernetes SIG met and determined that the next best outcome of the group is to create definitive documentation that the current contributors to the existing operators can get behind as a clear guide to Cassandra on Kubernetes that is published on the cassandra.apache.org site.

 

Outline / Collaboration Google Doc : [https://docs.google.com/document/d/15sZhAL-m9-a-6iuWmXdyKxsGraaJY9M6_xf9Qj2gSw4/edit?userstoinvite=jim.dickinson@datastax.com&ts=5f5a72f7]
Bug: Verify Correct Ownership of Attached Locations on Disk at Startup
Status: Triage Needed
Priority: Normal
Created: 09/Sep/20 20:49

There are two problematic scenarios around disk ownership we'll try to address here:

1.) A node comes up with an incorrectly mounted volume attached as its configured data directory. This causes the wrong system tables to be read.

2.) In a JBOD setup, the non-system keyspaces may reside on a volume separate from the system tables. In this scenario, we want to ensure that all directories belong to the same node, and that as the node starts up it can access all the directories it expects to be able to (including data, commit log, hints, and saved caches).

One solution to this is to touch an empty file in each relevant directory with a token that maps to the node to which it belongs. This would be done as part of the provisioning process during cluster creation or host replacement. A {{StartupCheck}} could then be added to verify that the correct identifying file is present in each directory, halting startup if not. It should, of course, be possible to disable this check (and it being disabled should likely be the initial default).

It's likely this will be most useful on EC2 or in k8s environments where these particular misconfigurations are likely to manifest.
Bug: Cassandra support add more thant one new nodes  at the same time 
Status: Triage Needed
Priority: Normal
Created: 08/Sep/20 04:33

We know that cassandra can not support more than  node bootstrap at one time, while the 

cassandra.consistent.rangemovement is true and

cassandra.consistent.simultaneousmoves.allow is false and if there exist one node be of

bootstrapping/leaving/moving status . If more thant one node doing boostrap, there may exist replica data being not consistent. But actually add new node one by one is time cost , especially when adding a new node will cost very long time. 

So we want to add more than one node at one time.
Bug: Add support in jvm dtest to test thrift
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 22/Jul/20 05:08
Resolved: 25/Jul/20 02:02

In 2.2 and 3.x thrift is supported and can cause problems for the non thrift code.  To make sure we can test these interactions, need to add support for thrift in jvm dtest.
Bug: cqlsh COPY functions broken in Python 3.8 on Mac
Status: Resolved
Priority: Normal
Resolution: Fixed
Created: 17/Aug/20 19:09
Resolved: 20/Aug/20 20:05

{noformat}
aholmberg-rmbp16:cassandra adamholmberg$ pyenv shell
3.8.2
aholmberg-rmbp16:cassandra adamholmberg$ bin/cqlsh --debug
Using CQL driver: <module 'cassandra' from '/Users/adamholmberg/code/cassandra/bin/../lib/cassandra-driver-internal-only-3.23.0.post0-1a184b99.zip/cassandra-driver-3.23.0.post0-1a184b99/cassandra/__init__.py'>
Using connect timeout: 5 seconds
Using 'utf-8' encoding
Using ssl: False
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 4.0-beta2-SNAPSHOT | CQL spec 3.4.5 | Native protocol v4]
Use HELP for help.
cqlsh> copy test.testcopyto to 'asdf';
Detected 12 core(s)
Using 11 child processes

Starting copy of test.testcopyto with columns [a, b, c, d].
Traceback (most recent call last):
  File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 937, in onecmd
    self.handle_statement(st, statementtext)
  File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 974, in handle_statement
    return custom_handler(parsed)
  File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 1563, in do_copy
    task.run()
  File "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 669, in run
    self.start_processes()
  File "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 471, in start_processes
    process.start()
  File "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object
cqlsh>
{noformat}


multiprocessing uses a different default start method on Mac, and pickling fails trying to serialize the Cluster object.
https://github.com/python/cpython/blob/db098bc1f05bd0773943e59f83489f05f28dedf8/Lib/multiprocessing/context.py#L313-L318
https://bugs.python.org/issue33725