opened
honzababarik
username_0
Hi,

I have an issue that might be very specific to my usecase and might not offer a workaround, but I thought I was gonna ask.
I am trying to have the Mongo DB run on Azure CosmosDB that supposed to be supporting MongoDB v5, but it does not seem like it does completely. When I try to run Agenda against a CosmosDB, the jobs querying fails:

`Multiple order-by items are not supported. Please specify a single order-by item.`

Trace:
`
MongoError: Message: {"Errors":["Encountered exception while executing function. Exception = Error: {\"Errors\":[\"Multiple order-by items are not supported. Please spe
cify a single order-by item.\"]}\r\nStack trace: Error: {\"Errors\":[\"Multiple order-by items are not supported. Please specify a single order-by item.\"]}\n   at queryCallback (__.sys.comm
onUpdate_BsonSchema.js:3580:26)\n   at Anonymous function (__.sys.commonUpdate_BsonSchema.js:617:29)"]}
16:28:29 1|fm2-jobs | ActivityId: aba98514-0000-0000-0000-000000000000, Request URI: /apps/cf7ce45a-ad86-401b-88b9-51807b88ba7a/services/254133b4-4c28-4b34-a771-6734147da565/partitions/22c9e
dd0-bbe7-4d8c-868e-4f40c07a04fb/replicas/131575688660948905p/, RequestStats: , SDK: Microsoft.Azure.Documents.Common/1.19.121.4
16:28:29 1|fm2-jobs |     at Function.MongoError.create (/opt/app/node_modules/mongodb-core/lib/error.js:31:11)
16:28:29 1|fm2-jobs |     at /opt/app/node_modules/mongodb-core/lib/connection/pool.js:497:72
16:28:29 1|fm2-jobs |     at authenticateStragglers (/opt/app/node_modules/mongodb-core/lib/connection/pool.js:443:16)
16:28:29 1|fm2-jobs |     at Connection.messageHandler (/opt/app/node_modules/mongodb-core/lib/connection/pool.js:477:5)
16:28:29 1|fm2-jobs |     at TLSSocket.<anonymous> (/opt/app/node_modules/mongodb-core/lib/connection/connection.js:331:22)`

When I have my own MongoDB instance, it all works just fine. I thought I was going to try to ask if someone knows if I can configure agenda to work and workaround using CosmosDB or I have to be running (and manage :-/) my own Mongo instance.

Thank you and big thanks for the amazing Agenda + Agendash pack!
Issue: Multiple order-by items not supported
issue
created
oscar-b
username_1
@username_0 Just ran into this issue as well, did you find any way around it?
comment
created
honzababarik
username_0
@username_1 Nope, don't have enough knowledge about Mongo internals to analyze the issue. From my basic understanding is the issue clearly on Azure not allowing multiple OrderBy in queries, so two workarounds would be to a) figure whether Agenda really needs two OrderBys for processing, and if not, make it configurable, b) start poking Microsoft into releasing a fix.

I decided to go for a self hosted Mongo solution on Azure since neither of those we are willing to invest into, though I am not happy about it. But the moment this works I am switching to Cosmos.
comment
created
oscar-b
username_1
This looks like it might be what we'd need, but it's only in planning stage: https://feedback.azure.com/forums/263030-documentdb/suggestions/16883608-allow-multi-order-by?category_id=321994
comment
created
oscar-b
username_1
Some more debug from Agenda:

```  agenda:define job [run stuff] defined with following options: 
  agenda:define { fn: [Function],
  agenda:define   concurrency: 5,
  agenda:define   lockLimit: 0,
  agenda:define   priority: 0,
  agenda:define   lockLifetime: 600000,
  agenda:define   running: 0,
  agenda:define   locked: 0 } +0ms
  agenda:database successful connection to MongoDB using collection: [agendaJobs] +0ms
  agenda:db_init init database collection using name [agendaJobs] +0ms
  agenda:db_init attempting index creation +1ms
  agenda:db_init index creation success +116ms
Agenda connected to MongoDB!
  agenda:start Agenda.start called, creating interval to call processJobs every [30000ms] +0ms
  agenda:internal:processJobs starting to process jobs +0ms
  agenda:internal:processJobs queuing up job to process: [run stuff] +0ms
  agenda:internal:processJobs job [run stuff] lock status: shouldLock = true +0ms
  agenda:_findAndLockNextJob _findAndLockNextJob(run stuff, [Function], cb) +0ms
  agenda:_findAndLockNextJob error occurred when running query to find and lock job +446ms
  agenda:internal:processJobs [run stuff] job lock failed while filling queue +447ms
```
comment
created
oscar-b
username_1
@username_0 It seems like it's the default sort order defining both `nextRunAt` and `priority` (https://github.com/agenda/agenda#sortquery) which causes the issue. Try setting sort to ```sort: { nextRunAt: 1 }```
comment
created
macroramesh6
username_2
I had the same issue with Azure cosmosDB.
comment
created
princjef
username_3
Ran into this issue as well when running atop CosmosDB. Setting the sort as described by @username_1 fixes the problem.
comment
closed
simison
username_4
issue