MongoDB
By: Mike Wolfe
Tags: Data Science, Database, Mongodb, NoSQL, Linux

MongoDB

Recap

In part one, I started building a database to use for a monthly

budgeting application. After installing MongoDB, we discussed

a brief overview of how to create a database, our first

collections, and inserting documents. With some general

knowledge about MongoDB, the goal is to finish building the remaining collections for the application, and to also learn more

about the syntax of MongoDB.

Continuation

Now that we have one collection created, we can look into understanding the many options that may be imposed on a collection. One such option is to add a maximum collection size to ensure data will not consume excessive space. By knowing more about these options, we can discover which will work best for the project.

Collection Options

Size: A rather straightforward option referring to the maximum bytes for a collection that may be capped. This is a number value.

Capped: This works with the size. Setting capped to true would enable MongoDB to overwrite the oldest document when the size capacity is reached. When false, documents cannot be overwritten. This is a Boolean value.

Max: Instead of the bytes-like in size, max refers to the number of documents allowed in a collection. Once again, this can be capped to override. This is a number value.

AutoIndexID: The most obvious option. MongoDB can either generate a primary key, similar to what many other database software does, or it may be omitted. This is a Boolean value.

Validator: Although a schema is not required with NoSQL, the validator option allows a JSON schema to be added to the collection. This allows for set fields and field types, as well as required fields. It applies only to inserts and deletes. This is a document value.

ValidationLevel: With the validator set, the validationlevel is a value to set have that validator set to “off” for no validations on inserts or updates, “moderate” which will force validation on all-new inserts and updates on valid documents but will not validate on any updates of existing invalid documents, or “strict” which will force validation for all inserts or updates. This is a string value.

ValidationAction: Now with the level of validation, an action must be selected. The action can be set to “warn” or “error”. For an “error”, the insert or update must follow the validation rules, otherwise, the document will not be written. A “warn” will allow writing whether validation is passed or failed, but if failed the error will be logged. This is a string value.

StorageOption: This option allows the user to specify where the collection is stored, such as on disk or in memory. This is a document value.

IndexOptionDefaults: If indexes were to be created for the collection, this option allows the user to set a default configuration for that index. This is a document value.

ViewOn: Instead of creating a view for a collection, the viewon option can be used to create one that will also use any indexes on the collection. The feature was added in MongoDB 3.4, so any previous versions will not include the option. This is a string value.

Pipeline: This option allows the user to supply the pipeline for the view, both in a createView and using the viewon option. It was also added in MongoDB 3.4. This is an array value.

Collation: The collation option allows the user to specify any language-specific rules for string comparisons. An example would be case sensitivity. It was added in MongoDB 3.4 so previous versions will not include the option. This is a document value.

WriteConcern: This option enables MongoDB to record errors if found during insertion. It is a document value.

Phew. Now that we covered a brief description of the collection options, we can get back to coding and maybe try a few options.

Collection Options

First, reconnect to mongo:

mongo

If the connection should refuse as mine did, you may need to simply restart:

sudo service mongod stop sudo service mongod start

To get started, we will create an example to get a feel for the new syntax. First, create a sample collection using the capped value. This will only be used for testing, as the capped option will not be used in the budgeting application. To use capped, there must be a size and max values as well:

db.createCollection(“ExampleCap”, {capped: true, size: 2, max: 1})

Recall the max option sets the number of documents allotted. In this case, the collection can hold only one. Now we can try to insert multiple values using a comma-separated list to see the outcome:

db.ExampleCap.insert({Name: “Insert One”}, {Name: “Insert Two”})

db.ExampleCap.find().pretty()

Now insert once more to find the old record is overwritten:

db.ExampleCap.insert({Name: “Insert Try Again”}) db.ExampleCap.find().pretty()

To remove the collection, simply use the drop function:

db.ExampleCap.drop()

One more example we will look at is with the validator option. This one will be usable, so it will be created on the Entertainment collection.

db.createCollection(“Entertainment”,{validator: {$jsonSchema: {required: [“Location”], properties: {Location: {bsonType: “string”], Total: {bsonType: “double”}, Giftcard: {bsonType: “double”}, CreatedDate: {bsonType: “string”}, TransactionDate: {bsonType: “date”}, Items: {bsonType: “array”}}}} validationLevel: “moderate”})

In the validator, we declared the JSON type Schema for the collection and required the Location. The other properties were then declared with their types. Now we can attempt to insert:

db.Entertainment.insert({Location: “GoodWill”, Total: 5.04, Giftcard: 0.00, CreatedDate: new Date(), TransactionDate: “2020-12-15”, Items: [{Name: “Lincoln Logs”, Price: 4.99}]})

Now try to insert without the location:

db.Entertainment.insert({Total: 4.23, Giftcard: 0.00, CreatedDate: new Date(), TransactionDate: “2020-12-15”, Items: [{Name: “Legos”, Price: 3.00}]})

The error is due to not having the required location. If we were to add a location, the record would then be inserted. As a default, the mistake caused an error. However, it could also be set to warn instead, which would allow the record to insert but with the error documented. In this case, this was only for an example, so we will not need to correct the data. Instead, we can add our remaining collections for the budget application. With Grocery and Entertainment, we will now need a Utility collection:

db.createCollection(“Utility”) db.Utility.insert({Type: “Electric”, Total: 151.92, CreatedDate: new Date(), WithdrawalDate: “2020-12-28”})

Next will be the Rent collection, which could cover rent or in our case mortgage:

db.createCollection(“Rent”) db.Rent.insert({Total: 2200, Minimum: 1070, Additional: 1130, CreatedDate: new Date(), TransactionDate: “2020-12-15”})

Almost there, we will need a collection for insurance:

db.createCollection(“Insurance”) db.Insurance.insert({Location: “Progressive”, Total: 102.78, CreatedDate: new Date(), TransactionDate: “2020-12-04”})

For the last one, we need a collection for emergency costs:

db.createCollection(“Emergency”) db.Emergency.insert({Total: 1000, CreatedDate: new Date(), TransactionDate: “2020-07-04”, Reason: “Water pump needed to be replaced.”})

We can now run a show command to see all the current collections:

Conclusion

To sum things up, we discussed several options for collections. We also implemented a few of those options as examples but used one in a collection that is a part of the project.

We viewed our current collections as Emergency, Entertainment, Grocery, Insurance, Rent, and Utility. These will be the expenses used in the budget application. For now, these will be our primary expenses. There may be more collections or even less tracked budgets per case basis, but for myself, these are what I would like to track.

Now that we have our collections flushed out and sample data has been added, we can begin creating the budgeting application. In the next part, we can delve deeper into how to track the data, and how we want the budgets to be represented. Until next time, cheers!