Repository: mongodb/laravel-mongodb (PHP)

Issue:
[Feature Request] Options to ignore auto conversion id to _id
### Is your feature request related to a problem?
We have a `merchant.id` field in our mongodb document, and after upgrading to mongodb driver 5.1 we encountered issue that mongodb driver modify the where query from `merchant.id` to `merchant._id` which results to unexpected query


### Describe the solution you'd like
It would be good if there's mechanism to whitelisting the fields from auto conversion `.id` to `._id`

### Describe alternatives you've considered
-

### Additional context
This part of code modify all `.id` to `._id`
<img width="931" alt="image" src="https://github.com/user-attachments/assets/c4aba8a0-aa18-4098-96e7-2134b6eed26b">



Discussion: I am looking for the solution here
    **protected $primaryKey = '_id';**
    I tried to put above line in model that is not working as well
    if we want to go with _id there must a be 
    please guide. 
> I am looking for the solution here

What I've done to get rid of the issue is by downgrading the MongoDB driver to version 4.x
We have no plan to allow usage of `.id` fields in embedded documents. For your existing database, you can rename the field with the `$rename` operator.

```php
DB::collection('products')->raw()->updateMany([], ['$rename' => ['merchant.id' => 'merchant._id']]);
```

If you need to keep both `id` and `_id` fields during update to avoid downtime, you can update in 3 steps:

1. Duplicate the field `merchant.id` into `merchant._id`
3. Deploy your code updating laravel-mongodb package
4. Remove `merchant.id`
That's too scary since we have a lot of services query the same collection and we don't want to break them.
We can still use the v4 version instead of renaming the field. I hope you guys have the bandwidth to support it in the future.
I'm also having an issue with this behavior. @GromNaN Would you consider making the `aliasIdForQuery` method protected, so that it could be overwritten? Renaming the nested id keys in our large code base is just too risky.
Could you provide more precision about your data model? Are you using `id` fields on the main document on only for embedded documents?
They are in embedded documents. In some places, they are structured as EmbedsMany models on a parent model, and in others, they are just arrays of objects stored on a model.
```
{
    "_id" : ObjectId(),
    "dynamic_content" : {
        "conditions" : [
            {
                "id" : "32C67020",
                "name" : "Deposit",
                "description" : "Deposited Students",
                "active" : true,
                "rules" : [
                    {
                        "content" : {...},
                        "type" : "user_segment"
                    }
                ]
            },
            {
                "id" : "6FED4F05",
                "name" : "Admit",
                "description" : "Admitted students-Nondeposits",
                "active" : true,
                "index_weight" : NumberInt(2),
                "rules" : [
                    {
                        "content" : {...},
                        "type" : "user_segment"
                    },
                    {
                        "content" : {...},
                        "type" : "date_condition"
                    }
                ]
            }
        ]
    }
}
```

The changes to alias '_id' in results and queries are incredibly problematic for us. We have almost 200 models in our code base. Our API handles about 3.5 million requests a day to ~1000 endpoints. Changing the structure of the data is not a reasonable option for us.

`aliasIdForResult` is public, so I was able to overwrite it, to continue returning '_id' alongside 'id'. But because `aliasIdForQuery` is private, I would need to overwrite `toMql`, `insert`, `insertGetId`, `delete`, `performUpdate`, and `inheritConnectionOptions` in order to change the behavior of `aliasIdForQuery` so that I can manage those models with nested 'id' values.
> We have no plan to allow usage of `.id` fields in embedded documents. For your existing database, you can rename the field with the `$rename` operator.

@GromNaN Is it known why subfields rename from "id" to "_id" happens in the first place? To be clear, I am talking about subfields in document, not the primary key "_id".

Another note, this change should have been in the release notes. Currently there is mentioned only the primary key.

All in all, it is problematic for my project as well and at least opt out would be lifesaver.
We plan to add a setting to disable `_id` conversion for embedded documents.
> We plan to add a setting to disable `_id` conversion for embedded documents.

It is superb to hear that! Is there a target version in mind as well or any other indication about timeline?
Hey,
Since this is a problem causing corruptions in data modeling and querying, it indeed must be considered a bug rather than a feature.
Another scenario I have encountered such problem is that, I have a document like the following stored in the database:
```json5
{
  "_id": ObjectId('abcd123'),
  "key": "value",
  "data": [
    0: {
      "content": "Lorem ipsum",
      "another_key": "another_value",
      "_id": "a unique ID from another storage",
      "id": "another unique ID from another storage"
    },
    1: {
      "content": "Lorem ipsum II",
      "another_key": "another_value_ii",
      "_id": "a unique ID from another storage II",
      "id": "another unique ID from another storage II"
    },
    ...
  ]
}
```
When I retrieve this document by Eloquent, all '_id' fields are converted into 'id', event those nested '_id' fields having nothing to do with primary-key. So it would be like this:
```json5
{
  "id": ObjectId('abcd123'),
  "key": "value",
  "data": [
    0: {
      "content": "Lorem ipsum",
      "another_key": "another_value",
      "id": "another unique ID from another storage"
    },
    1: {
      "content": "Lorem ipsum II",
      "another_key": "another_value_ii",
      "id": "another unique ID from another storage II"
    },
    ...
  ]
}
```

After realizing it, I encountered a worse case which was overriding `protected $primaryKey = 'id';` to use '_id'. _However, nothing affected the final result._
This _id to id auto conversion is so awkward. I understand the intention and that the maintainers want to unify the package with Laravel way, but introducing such a huge change without any backward care is a huge pain!

Our frontend, API, and Wehbook listeners expect to receive the _id field, now, after upgrading to version 5.x, they all receive `id` instead, and there is no way to return it to `_id`. imagine how many applications will that break, and how hard to tell all these parties that they should listen for the `id` from now on. also it would be unjustifiable change.

would appreciate having a way to stop this conversion, as the features introduced in version 5 are amazing, and we would love to upgrade to v5 without weeks of work and downtime!
Thanks all for your feedback. 
We plan to start working on [PHPORM-255](https://jira.mongodb.org/browse/PHPORM-255) that addresses this challenge in the next couple of weeks.
@bisht2050 
PHPORM-255 seems to suggest to allow whitelisting fields that opt out from the conversion. What most here are looking for is disabling the conversion fully. 
@leitommi the issue copies the description of this pull request, and only links a pull request received from an external contributor (now closed) for reference.

At the moment, we're favouring an option to disable this automatic conversion for embedded documents. Embedded documents don't necessarily need an ID (but can profit from having one), so we shouldn't mess around with the identifier in embedded documents. However, since we have made the hard BC break for this, the only way away from this is to keep the BC break in 5 (to avoid having to release version 6 with another BC break), but to allow people to opt out from this new behaviour. Essentially, updating to version 5 of the integration requires you to update this.

All in all, we're suffering from an old design decision made here. MongoDB always has used `_id` as a primary key, and there was never a way to disable this or change this. Since the value of this `_id` field can be anything, a compound primary key would be represented as an array or an embedded document, even though MongoDB recommends doing that because of potential issues when querying.

Laravel on the other hand has always used `id` as the primary key. If we were to rewrite this library from scratch, we would always want people to use `id` in their models, but map it to the `_id` field in the documents, as that's what satisfies both conventions best. Since software development usually isn't a green field, we're stuck with some decisions made long before we took over the library, and we made a judgment call to change this behaviour without fully understanding the implications and the impact it has on the users.

There's this example from earlier which I want to address:
```json
{
  "content": "Lorem ipsum",
  "another_key": "another_value",
  "_id": "a unique ID from another storage",
  "id": "another unique ID from another storage"
}
```

I hope for your sanity that this is just an example designed to highlight the issue, not an actual use case. Having `_id` and `id` with two separate values is asking for trouble - simply because they're too easy to confuse. If I were to have the use case of needing my own identifier as well as storing an identifier from an external system, I'd use `_id` for my own identifier (as that's the convention) and some sort of `externalId` (or even better named for the appropriate system) for the external identifier. However, I'm not going to tell you how to design your documents. As hair raising as that example may be, we want to give users a way to keep working with these documents.

We will however, keep the current behaviour for root documents: when people use `id`, they usually mean the identifier of the document, which in MongoDB is `_id`. One simple reason for this change is that a lot of Laravel functionality assumes the primary key field to be `id`, even when you define something else in the model. We've previously argued for changing this (see https://github.com/laravel/framework/pull/48089 and https://github.com/laravel/framework/pull/52079), but the change was denied both times. So, the only way for us to allow people to use Laravel components with MongoDB was to assume that `id` refers to `_id`. Anything other than that would've involved rewriting parts of Laravel specifically for MongoDB, making the developer experience much worse as you'd need separate paginators for something that should be abstracted away by Eloquent.
> we made a judgment call to change this behaviour without fully understanding the implications and the impact it has on the users.

this admission has honestly given me some peace lol. thank you for addressing this issue.

i spent weeks upgrading our system to v5, and ended up overwriting a lot of the new code to avoid these conversions. but if i had been able to opt out of changing nested _id keys, i might have been able to get away with a simpler solution. so i'm sharing it here in case it helps anyone else.

You can use eloquent's $appends property to ensure '_id' is set on all of your serialized models. And then extend all of your models from the adjusted base model class. 
```
<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Casts\Attribute;
use Illuminate\Database\Eloquent\Model;

class BaseModel extends Model
{
    /**
     * The accessors to append to the model's array form.
     *
     * @var array
     */
    protected $appends = ['_id'];

    /**
     * Append _id to arrays
     */
    protected function id(): Attribute
    {
        return new Attribute(
            get: fn (string $value) => $value ?? $this->id,
        );
    }
}
```
This way, you don't have to find every place in your code where you expected '_id' to exist on an array. You will still have to deal with the addition of the new 'id' property, and hide it in places where you don't want to expose the primary key.
I drafted a PR, I welcome your feedbacks: https://github.com/mongodb/laravel-mongodb/pull/3332



Fix (patch):
diff --git a/src/Connection.php b/src/Connection.php
index 4dd04120d..29b72ae44 100644
--- a/src/Connection.php
+++ b/src/Connection.php
@@ -53,6 +53,9 @@ class Connection extends BaseConnection
 
     private ?CommandSubscriber $commandSubscriber = null;
 
+    /** @var bool Whether to rename the rename "id" into "_id" for embedded documents. */
+    private bool $renameEmbeddedIdField;
+
     /**
      * Create a new database connection instance.
      */
@@ -80,6 +83,8 @@ public function __construct(array $config)
         $this->useDefaultSchemaGrammar();
 
         $this->useDefaultQueryGrammar();
+
+        $this->renameEmbeddedIdField = $config['rename_embedded_id_field'] ?? true;
     }
 
     /**
@@ -395,6 +400,18 @@ public function __call($method, $parameters)
         return $this->db->$method(...$parameters);
     }
 
+    /** Set whether to rename "id" field into "_id" for embedded documents. */
+    public function setRenameEmbeddedIdField(bool $rename): void
+    {
+        $this->renameEmbeddedIdField = $rename;
+    }
+
+    /** Get whether to rename "id" field into "_id" for embedded documents. */
+    public function getRenameEmbeddedIdField(): bool
+    {
+        return $this->renameEmbeddedIdField;
+    }
+
     /**
      * Return the server version of one of the MongoDB servers: primary for
      * replica sets and standalone, and the selected server for sharded clusters.
diff --git a/src/Eloquent/Builder.php b/src/Eloquent/Builder.php
index aabc526f7..f3ffd7012 100644
--- a/src/Eloquent/Builder.php
+++ b/src/Eloquent/Builder.php
@@ -18,6 +18,7 @@
 use MongoDB\Model\BSONDocument;
 
 use function array_key_exists;
+use function array_map;
 use function array_replace;
 use function collect;
 use function is_array;
@@ -237,7 +238,7 @@ public function raw($value = null)
         // Convert MongoCursor results to a collection of models.
         if ($results instanceof CursorInterface) {
             $results->setTypeMap(['root' => 'array', 'document' => 'array', 'array' => 'array']);
-            $results = $this->query->aliasIdForResult(iterator_to_array($results));
+            $results = array_map(fn ($document) => $this->query->aliasIdForResult($document), iterator_to_array($results));
 
             return $this->model->hydrate($results);
         }
diff --git a/src/Query/Builder.php b/src/Query/Builder.php
index 07a3483b0..7d0fdce74 100644
--- a/src/Query/Builder.php
+++ b/src/Query/Builder.php
@@ -29,6 +29,7 @@
 use MongoDB\Builder\Type\SearchOperatorInterface;
 use MongoDB\Driver\Cursor;
 use MongoDB\Driver\ReadPreference;
+use MongoDB\Laravel\Connection;
 use Override;
 use RuntimeException;
 use stdClass;
@@ -83,6 +84,7 @@
 use function trait_exists;
 use function var_export;
 
+/** @property Connection $connection */
 class Builder extends BaseBuilder
 {
     private const REGEX_DELIMITERS = ['/', '#', '~'];
@@ -1764,9 +1766,9 @@ public function orWhereIntegerNotInRaw($column, $values, $boolean = 'and')
         throw new BadMethodCallException('This method is not supported by MongoDB');
     }
 
-    private function aliasIdForQuery(array $values): array
+    private function aliasIdForQuery(array $values, bool $root = true): array
     {
-        if (array_key_exists('id', $values)) {
+        if (array_key_exists('id', $values) && ($root || $this->connection->getRenameEmbeddedIdField())) {
             if (array_key_exists('_id', $values) && $values['id'] !== $values['_id']) {
                 throw new InvalidArgumentException('Cannot have both "id" and "_id" fields.');
             }
@@ -1793,7 +1795,7 @@ private function aliasIdForQuery(array $values): array
             }
 
             // ".id" subfield are alias for "._id"
-            if (str_ends_with($key, '.id')) {
+            if (str_ends_with($key, '.id') && ($root || $this->connection->getRenameEmbeddedIdField())) {
                 $newkey = substr($key, 0, -3) . '._id';
                 if (array_key_exists($newkey, $values) && $value !== $values[$newkey]) {
                     throw new InvalidArgumentException(sprintf('Cannot have both "%s" and "%s" fields.', $key, $newkey));
@@ -1806,7 +1808,7 @@ private function aliasIdForQuery(array $values): array
 
         foreach ($values as &$value) {
             if (is_array($value)) {
-                $value = $this->aliasIdForQuery($value);
+                $value = $this->aliasIdForQuery($value, false);
             } elseif ($value instanceof DateTimeInterface) {
                 $value = new UTCDateTime($value);
             }
@@ -1824,10 +1826,13 @@ private function aliasIdForQuery(array $values): array
      *
      * @template T of array|object
      */
-    public function aliasIdForResult(array|object $values): array|object
+    public function aliasIdForResult(array|object $values, bool $root = true): array|object
     {
         if (is_array($values)) {
-            if (array_key_exists('_id', $values) && ! array_key_exists('id', $values)) {
+            if (
+                array_key_exists('_id', $values) && ! array_key_exists('id', $values)
+                && ($root || $this->connection->getRenameEmbeddedIdField())
+            ) {
                 $values['id'] = $values['_id'];
                 unset($values['_id']);
             }
@@ -1837,13 +1842,16 @@ public function aliasIdForResult(array|object $values): array|object
                     $values[$key] = Date::instance($value->toDateTime())
                         ->setTimezone(new DateTimeZone(date_default_timezone_get()));
                 } elseif (is_array($value) || is_object($value)) {
-                    $values[$key] = $this->aliasIdForResult($value);
+                    $values[$key] = $this->aliasIdForResult($value, false);
                 }
             }
         }
 
         if ($values instanceof stdClass) {
-            if (property_exists($values, '_id') && ! property_exists($values, 'id')) {
+            if (
+                property_exists($values, '_id') && ! property_exists($values, 'id')
+                && ($root || $this->connection->getRenameEmbeddedIdField())
+            ) {
                 $values->id = $values->_id;
                 unset($values->_id);
             }
@@ -1853,7 +1861,7 @@ public function aliasIdForResult(array|object $values): array|object
                     $values->{$key} = Date::instance($value->toDateTime())
                         ->setTimezone(new DateTimeZone(date_default_timezone_get()));
                 } elseif (is_array($value) || is_object($value)) {
-                    $values->{$key} = $this->aliasIdForResult($value);
+                    $values->{$key} = $this->aliasIdForResult($value, false);
                 }
             }
         }