Repository: yougov/mongo-connector

Issue #643: Connector exiting with --no-dump
Created: 2017-02-02T21:13:02Z
Hi,
We're implementing our own doc manager that send data to a stream. When implementing just the remove and upsert functions work fine, but when implementing the update function it seems having issues.

remove and upsert operations are about 30 operation per second
update goes up to 1000 operations per sec

To describe the issue :
mongo-connector -c config.json --no-dump
After starting it, it sends about 90 records before exiting with :
OplogThread: Last entry no longer in oplog cannot recover!

Then :
mongo-connector -c config.json
It directly exits with the same error (without processing any record)
The connector works fine when we remove the line sending data from the update function to the stream.
The wiki give us some indications on resyncing the connector, but this not solve the issue.
I was thinking it could have a kind of time out going on because we might have too many operations in the update function? Or would that be something else? Any advices on it? Thank you for shedding some lights :)

Duplicate issue #606: mongo_connector errors on large dbs
Created: 2016-12-12T12:39:14Z
source cluster mongodb:2.6.11
destination cluster mongodb:v3.2.11
mongo-connector:v2.5.0dev0
The question:
I use mongo-connector to sync data from mongodb 2.6.12 to mongodb 3.2.11.
All cluster's have 3 shards. The collections has about 74,753,772 docs. Mongo-connector started initially sync. And after finish this process, starting processing oplog. In logs i see that every 30 sec counter always increases for shard rs01 that have biggest part of data.
After 3 days mongo-connector stopped with error:
...
2016-12-11 06:26:27,657 [INFO] mongo_connector.oplog_manager:72 - OplogThread for replica set 'rs03' is up to date with the oplog.
2016-12-11 06:26:28,718 [INFO] mongo_connector.oplog_manager:62 - OplogThread for replica set 'rs02' is 9 seconds behind the oplog.
2016-12-11 06:26:30,584 [INFO] mongo_connector.oplog_manager:62 - OplogThread for replica set 'rs01' is 193097 seconds behind the oplog.
...
...
...
2016-12-12 11:27:47,341 [INFO] mongo_connector.oplog_manager:72 - OplogThread for replica set 'rs03' is up to date with the oplog.
2016-12-12 11:27:51,058 [INFO] mongo_connector.oplog_manager:62 - OplogThread for replica set 'rs01' is 297578 seconds behind the oplog.
2016-12-12 11:27:51,326 [INFO] mongo_connector.oplog_manager:72 - OplogThread for replica set 'rs02' is up to date with the oplog.
2016-12-12 11:28:17,371 [INFO] mongo_connector.oplog_manager:72 - OplogThread for replica set 'rs03' is up to date with the oplog.
2016-12-12 11:28:21,089 [INFO] mongo_connector.oplog_manager:62 - OplogThread for replica set 'rs01' is 157174 seconds behind the oplog.
2016-12-12 11:28:21,347 [INFO] mongo_connector.oplog_manager:62 - OplogThread for replica set 'rs02' is 34 seconds behind the oplog.
2016-12-12 11:28:29,878 [ERROR] mongo_connector.oplog_manager:336 - OplogThread: Last entry no longer in oplog cannot recover! Collection(Database(MongoClient(host=[u'mongodb-rs01-02:27018', u'mongodb-rs01-03:27018', u'mongodb-rs01-01:27018'], document_class=dict, tz_aware=False, connect=True, replicaset=u'rs01'), u'local'), u'oplog.rs')
2016-12-12 11:28:30,212 [ERROR] mongo_connector.connector:368 - MongoConnector: OplogThread <OplogThread(Thread-2, stopped 140200225036032)> unexpectedly stopped! Shutting down
2016-12-12 11:28:30,212 [INFO] mongo_connector.connector:406 - MongoConnector: Stopping all OplogThreads
2016-12-12 11:28:38,986 [INFO] mongo_connector.doc_managers.mongo_doc_manager:137 - Mongo DocManager Stopped: If you will not target this system again with mongo-connector then you may drop the database __mongo_connector, which holds metadata for Mongo Connector.

Why oplog processing is quite slowly?
Thanks in advance!

Closing comment (2017-03-01T00:48:30Z): I'm curious if this one could be related to #643 ? We're connecting mongo-connector to 8 shards separately, the 1st shard( that seems to have the most data) ofter exits with
OplogThread: Last entry no longer in oplog cannot recover!