Issue 1:
I'm not sure this is a bug because the error message seems to be arising at the conditional

neo4j/community/record-storage-engine/src/main/java/org/neo4j/internal/batchimport/RelationshipGroupCache.java

Line 201
in
4fe5e8b

"Tried to put multiple groups with same type " + type + " for node " + owningNodeId );

IMPORT FAILED in 44m 50s 527ms.
3559494 nodes
500726004 relationships
4997255236 properties
Peak memory usage: 652.6MiB
Import error: Tried to put multiple groups with same type 5 for node 2506441
Caused by:Tried to put multiple groups with same type 5 for node 2506441
java.lang.IllegalStateException: Tried to put multiple groups with same type 5 for node 2506441
at org.neo4j.internal.batchimport.RelationshipGroupCache.scanForFreeFrom(RelationshipGroupCache.java:200)
at org.neo4j.internal.batchimport.RelationshipGroupCache.put(RelationshipGroupCache.java:170)
at org.neo4j.internal.batchimport.CacheGroupsStep.process(CacheGroupsStep.java:54)
at org.neo4j.internal.batchimport.CacheGroupsStep.process(CacheGroupsStep.java:33)
at org.neo4j.internal.batchimport.staging.ProcessorStep.lambda$receive$1(ProcessorStep.java:84)
at org.neo4j.internal.batchimport.executor.DynamicTaskExecutor$Processor.run(DynamicTaskExecutor.java:220)
at java.base/java.lang.Thread.run(Thread.java:829)

However I am at a loss at what is causing the issue and since there's ~70GB of the relationship TYPE I was trying to import I'm not sure where to start other than the Exception message.

Neo4j Version: Docker- neo4j:4.3.7

Operating System: Ubuntu 20.04.3 LTS

RAM: 62.7 GiB

Processor: AMD® Ryzen 9 3900xt 12-core processor × 24

API: Docker

Using admin import (node and relationship lines removed in code below):

docker run --interactive --rm \
--publish=7474:7474 --publish=7687:7687 \
--volume="${1}/data:/var/lib/neo4j/data" \
--volume="${1}/import:/var/lib/neo4j/import" \
--volume="${1}/plugins:/var/lib/neo4j/plugins" \
--volume="${1}/import.report:/var/lib/neo4j/import.report" \
--user=$(id -u):$(id -g) \
neo4j:4.3.7 \
neo4j-admin import \
... \
--delimiter="\t" \
--high-io=true \
--processors=20 \
--database=neo4j \
--ignore-empty-strings=true \
--ignore-extra-columns=true \
--skip-bad-relationships \
--skip-duplicate-nodes

Currently running import again with ~half the data of that relationship type...

Issue 2:
We have upgraded our Neo4j instances from 4.1.3 to 4.2.2 and since then MERGE no longer prevents duplication of Nodes while concurrent queries are being executed. It is as if the MERGE operation can no longer hold a write lock with the specified condition.

Example query that trigger the problem:

MERGE (s:Tenant:Service{identity:'683b0e50-debf-4e69-b3df-8b22b4068f4c'})
SET s.name = 'myservice'
WITH *
UNWIND [{ identity: '94b96bd1-63f1-4f11-876a-d36ddc23d42d', pipelineName: 'pipeline1' }, { identity: '1098ae63-0148-48b7-a3a1-97706c47f8a5', pipelineName: 'pipeline2' }, { identity: 'a17d96f5-53e9-4648-a56b-3cb280f0f252', pipelineName: 'pipeline3' }] as pipeline
MERGE (s)-[:provides]->(p:Tenant:Pipeline{identity:pipeline.identity})
SET p.name = pipeline.name
WITH *
RETURN p

I am not sure what changed, but since our upgrade it can be consistently reproduced in every single attempt. If we try to reproduce using 4.1.3, it never happens. We have a quick way to launch a clean environment with either of the versions of Neo4j. And for our scenario it's not possible to create an index at the moment these queries are executed and until now, it hasn't been a problem.

The source of the queries are replicated NodeJS applications in a Kubernetes clusters. We run a standalone instance of Neo4j so all replicas execute those queries to the same Neo4j core server.

Additional information:

Neo4j version: 4.2.2 Enterprise

Operating system: neo4j enterprise docker image

API/Driver: javascript driver. Tried with both 4.0.2 and 4.2.2 versions.

Steps to reproduce:

Unfortunately there is no easy way to provide the reproducible environment as it involves our code and infrastructure. I am assuming Neo4j team is able to create parallel execution scenarios. Using the provided query should be enough to reproduce as we can reproduce it in a blank database.

Please let me know if there is anything else that could help determine the problem or if you're having issues to reproduce the problem.

Expected behavior:

Merge prevents the duplication of the Pipeline node that is created in this section of the cypher query

MERGE (s)-[:provides]->(p:Pipeline{identity:pipeline.identity})

Actual behavior

Usually end up having on set of duplicated Pipeline nodes (same value for identity property)