I have an exam tomorrow and I want to study the following pdf. My exam only contains the theorical parts of this pdf, and not the coding part. I want you to write a comprehensive and structured recap out of this pdf file in a way to be easily memorizable. 

Introduction to
Cloud Firestore
AA. 2022-23

Introduction to Cloud Firestore
• The course project will rely, for its backend on the Cloud
Firestore database, part of Google Firebase platform
o https://firebase.google.com/
o Document based, NoSQL backend as a service (BaaS)
o Owned and supported by Google
o Can be used both for small and for large applications
o Various pricing options, free for small projects
• Versatile and proven platform with many bindings
o Android, iOS, Javascript, REST,…©G. Malnati, J. Xhembulla 2016-23 3
NoSQL
• Broad term that refers to any DBMS that does not rely on the
SQL language for querying data
o NoSQL DBMSs typically provide support for many other ways to store data,
not necessarily tables made of simple tuples
• Designed for spreading over several server machines
o In order to scale both in terms of number of records and in terms of
number of concurrent users
• Scalability comes at a price: consistency is eventual
o Data is not immediately updated
o Consequence of the C,A,P theorem©G. Malnati, J. Xhembulla 2016-23 4
Types of NoSQL DBMS
• Key/Value stores
o Big distributed hash-table
o Each data item has a unique key
o Actual data is not understood by the server
o Examples: Redis, Riak
• Document stores
o Similarly to Key/Value, each data
item has a unique key
o Data is a structured document,
often represented in JSON format
o Examples: MongoDB, CouchDB©G. Malnati, J. Xhembulla 2016-23 5
Types of NoSQL DBMS
• Column data stores
o Single records represent data columns,
not data rows as in relational DBs
o This approach simplifies data aggregation
o Examples: Cassandra, Hbase
• Graph stores
o Data is organized in graphs, using
nodes and links
o Examples: Neo4J, HyperGraphDB,
JanusGraph©G. Malnati, J. Xhembulla 2016-23 6
Cloud Firestore architecture
• Schema-less document store
o Data is organized in collections and documents
• Every collection has a name
o A collection may contain zero or more documents
• A document consists of a JSON data structure
o Having textual keys and boolean, numerical, textual, temporal, geographical
values
o Values can also consist of list of values, sub-objects (maps), or references to
other objects (caution!)
o Documents can contain sub-collection, too
o A document is limited to 1MB in size, whatever its internal structure
• Every document has a ID
o Assigned by the system or by the programmer
o ID is unique within the collection where the document is stored, and is part of
its path©G. Malnati, J. Xhembulla 2016-23 7
Expressive querying
• Firestore allows to retrieve individual documents inside a
collection
o Provided their id is known
• …or to retrieve all documents of the collection that match a set
of constraints
o Queries can contain multiple filters and sorting criteria
o Thanks to indexing, query performance depends on the result set size
and not on the data set size
users
u1: { name: “john”, role: “customer”, language: “en” }
u2: { name: “mary”, role: “admin”, language: “en” }
u3: { name: “bot”, role: “bot”, enabled: true }
Collection
ID Document
Documents in a collection
can have different structures©G. Malnati, J. Xhembulla 2016-23 8
Realtime updates
• Firestore sets up a data synchronization channel between the
database and each connected device
o Whenever the data set changes, a notification is sent to each device in
order to synchronize
o Queries can be expressed as one-time fetch only, in order to improve
overall efficiency
• Queried data is cached automatically
o Thus allowing offline operations
o Both read and write access is possible when the device is offline: as
soon as it get connected again, data is synchronized again with the
main store©G. Malnati, J. Xhembulla 2016-23 9
Modelling data
• A document consists of named pieces of data
{
"title" : "Mobile Application Development",
"groups" : {
"g1" : [ "name1", "name2", "name3", "name4" ],
"g2" : [ "name5", "name6", "name7", "name8" ]
},
"tableOfContents" : ["Kotlin", "Android", "Firestore"
"User Centered Design", "Hybrid Apps" ],
"year" : 2022
}©G. Malnati, J. Xhembulla 2016-23 10
Collections
• A complete database can be as simple as a single document
o Or it may contain thousands of collections each storing a huge amount of
documents
• Document may have any structure and a single collection can contain
heterogenous documents
o But, usually, a more conservative approach is chosen, putting similar
documents inside single collections
• Every document in Cloud Firestore is uniquely identified by its location
within the database
o A reference is a lightweight object that points to a location in the database,
whether it exists or not
o val mad =db.collection("courses").document("mad")
val alwaysMad = db.document("courses/mad")©G. Malnati, J. Xhembulla 2016-23 11
Sub-collections
• A given document might contain many sub-items
o Although it is possible to store them inside the document itself, usually this
is not a good idea
• Single documents have size limitations
o They must be smaller than 1Mbyte
o Have no more than 20,000 properties
o Have no more than 20 level deep nesting
• A sub-collection is a collection associated with a specific
document©G. Malnati, J. Xhembulla 2016-23
Sub-collections
12
courses
mad
students
student1
student2
wa2
students
student3©G. Malnati, J. Xhembulla 2016-23 13
Sub-collections
• References to items in sub-collections can be created easily
o val student1 = db
.collection("courses")
.document("mad")
.collection("students")
.document("student1")
• Sub-collections can be nested up to 100 levels deep©G. Malnati, J. Xhembulla 2016-23 14
Data structure comparison
• Nested data in documents
o Good when simple, fixed list of data need to be kept together, in order to
speed up retrieval
o If data becomes large, load time increases and upper limits may be reached
• Sub-collections
o Allow easy querying of sub-items and let the parent document size unaffected
by the number of items
o Difficult to delete: if the parent document is deleted, the sub-collection stays
there
• Root-level collections
o Extremely flexible and scalable
o Difficult to manage relationships between items belonging to sibling collections©G. Malnati, J. Xhembulla 2016-23 15
Modelling relationships
• If a given data item is related to other items, two modelling
approaches are possible:
o Referencing data in other collections
o De-normalising data
Supplier Component
Product
n
m
m n©G. Malnati, J. Xhembulla 2016-23 16
Referencing data
• A document reference is a kind of URL
o As such, it can be stored in the source document in order to allow the
retrieval of the target document
• A new collection must be created to implement many-to-many
relationships
o Its documents contain references to two related parties
• Not having columns, Firestore does not support column joins
o Two requests need to be issued: one for getting the set of relationship
documents, the second to get the related items
o The first request implies a search (an index on each referencing field need
to be built)
• Firestore does not support cascading
o If one of the documents is deleted, the other keeps on referencing it, no
way to propagate automatically the deletion©G. Malnati, J. Xhembulla 2016-23 17
De-normalising data
• As an alternative, a data duplication may be introduced
o Products, components and suppliers can be stored in separate collections
o Each supplier can have an array field listing the set of components it
produces, and each product has an array field with the set of components
it is made of
o Each component has two array fields: one listing the set of possible
suppliers, the other listing the set of products that use it
• De-normalization introduces redundancy and possible conflicts
o What happens if only one side of the relationship is updated?
o If two pieces of information that should be equal are different, which one
must be trusted?©G. Malnati, J. Xhembulla 2016-23 18
The Firebase console
• Allows the programmer to create, configure, and modify the
structure, the access rules and the behaviour of a Firebasepowered application
o A Google account is required
o https://console.firebase.google.com/
• From here a new Cloud Firestore database can be created
o As well as authentication rules, data-storage, back-end functions, and
machine learning©G. Malnati, J. Xhembulla 2016-23 19
The Firebase assistant
• A convenient support providing
guidance when configuring and
using Firebase products
o AndroidStudio
→ Tools
→ Firebase©G. Malnati, J. Xhembulla 2016-23 20
Cloud Firestore and Android
• A complete library is available for accessing a Cloud Firestore
database from an Android application
o It allows programmer to query and update data in a seamless way
• Full instructions for configuring the project can be found here
o https://firebase.google.com/docs/android/setup
o At the end of the process, a google-services.json file will be generated and
put inside the android project©G. Malnati, J. Xhembulla 2016-23 21
Accessing Firestore
• All operations involving the database start with an object of class
FirebaseFirestore
o val db = FirebaseFirestore.getInstance()
o Typically stored as an activity property or a property of a singleton object
o The database configuration is automatically derived from the
google-services.json file
• References to collections and objects are created via
o val coll1 = db.collection("courses")
o val doc1 = db.collection("courses")
.document("mad")
o val doc2 = db.document("courses/mad")
• All reading and writing operations are asynchronous
o Callbacks are used to get results or failure indications©G. Malnati, J. Xhembulla 2016-23 22
Modelling data
• In Kotlin, data can be stored in maps
o Or in a corresponding data class with a default constructor
val course = HashMap<String, Any>()
course["title"] = "Mobile Application Development"
course["groups"] = hashMapOf<String, Any>(
"g1" to listOf("name1", "name2", "name3", "name4"),
"g2" to listOf("name5", "name6", "name7", "name8")
)
course["tableOfContents"] = listOf(
"Kotlin", "Android", "Firestore",
"User Centered Design", "Hybrid Apps")
course["year"] = 2022©G. Malnati, J. Xhembulla 2016-23 23
Modelling data
• Data can also be modelled via data classes
o Annotations may be added to instruct the serialization process
data class Course (
var title: String = "",
var groups: Map<String, List<String>> = mapOf(),
@set:PropertyName("tableOfContents")
@get:PropertyName("tableOfContents")
var toc: List<String> = emptyList(),
var year: Int = 0
}©G. Malnati, J. Xhembulla 2016-23
Writing to the database
24
db
.collection("courses")
.document("mad2022")
.set( modelData() ) //the chosen representation
.addOnSuccessListener {
Toast
.makeText(this, "Data saved", Toast.LENGTH_SHORT)
.show()
} .
addOnFailureListener{
Toast
.makeText(this, "Error", Toast.LENGTH_LONG)
.show()
}©G. Malnati, J. Xhembulla 2016-23
Reading from the database
25
db
.collection("courses")
.document("mad2021")
.get()
.addOnSuccessListener {
res ->
val course =
res.toObject(Course::class.java)
//use it as needed
} .
addOnFailureListener {
Toast
.makeText(this, "Error", Toast.LENGTH_LONG)
.show()
}©G. Malnati, J. Xhembulla 2016-23 26
Kotlin type mapping
• Each elementary value stored in the Firestore JSON object is
mapped to the corresponding Kotlin class
o JSON numbers becomes either Long or Double instances, according to their
decimal part
o JSON strings are mapped to Kotlin Strings
o Dates are mapped to Timestamp
o Boolean literals are mapped to Boolean instances
o JSON objects are mapped to Map<String, Any>
o JSON lists are mapped to List<Any>
• The toObject(c: Class) method can be used to have the returned
data mapped to a data class©G. Malnati, J. Xhembulla 2016-23 27
Listening to real time updates
• It is possible to subscribe to a single document to intercept any
change it may undergo
o A SnapshotListener is invoked as soon as the document is subscribed to
and whenever it changes
o Each time it is invoked, the listener will receive a copy of the most recent
document content
val docRef = db.collection("courses").document("mad2020")
val reg = docRef.addSnapshotListener { snapshot, e ->
if (e != null) {
Log.w(TAG, "Listen failed.", e)
return@addSnapshotListener
}
if (snapshot != null && snapshot.exists()) {
Log.d(TAG, "Current data: ${snapshot.data}")
} else {
Log.d(TAG, "Current data: null")
}
}©G. Malnati, J. Xhembulla 2016-23 28
Listening to real time updates
• If a query is specified, it is possible to detect any change to the
result set
o A document is added, removed, or modified
val registration = db.collection("courses")
.whereEqualTo("year",2022)
.addSnapshotListener{ value, e ->
if (e != null) { //Handle error
} else {
val courses = mutableListOf<Course>()
for (doc in value!!) courses.add(
doc.toObject(Course::class.java) )
} //use courses somewhere
}©G. Malnati, J. Xhembulla 2016-23 29
Deleting a listener
• If the app is no more interested in subscribed data (e.g., because
the activity / fragment is quitting) the listener should be
cancelled
o The object returned by addSnapshotListener(…) offers a remove() method
to this purpose
• If an error arises, the listener will be invoked with a non-null
error argument
o After an error, the listener will not receive any more events, and there is
no need to detach the listener©G. Malnati, J. Xhembulla 2016-23 30
Querying data
• A document or collection reference object offers a set of
methods to perform data queries
o whereEqualsTo(field, value)
o whereGreaterThan(field, value)
o whereGreaterThanOrEqualTo(field, value)
o whereLessThan(field, value)
o whereLessThanOrEqualTo(field, value)
o whereIn(field, valueList)
o whereArrayContains(field, value)
o whereArrayContainsAny(field, valueList)
• They all return a Query object
o Which can be read (via get()), subscribed to, or further constrained via
where… methods©G. Malnati, J. Xhembulla 2016-23 31
Query limitations
• Queries with range filters on different fields are not supported
• Queries with a != clause are not supported
o The query can be split into a greater-than query and a less-than query
• Cloud Firestore provides limited support for logical OR queries
o The in and array-contains-any operators support a logical OR of up to 10
equality (==) or array-contains conditions on a single field
o For other cases, create a separate query for each OR condition and merge
the query results in your app©G. Malnati, J. Xhembulla 2016-23 32
Ordering and limiting data
• Method .orderBy(fieldname) allows to specify the sort
order of the retrieved data
o As well as act as a filter, removing all document that lacks the specified
field
o By default, ascending order is applied
• Method .limit(size) sets an upper bound on the number of
returned documents
o Methods .startAt(position) and .endAt(position) allows to set a
range of results to be retrieved
o Similarly to .startAfter(position) and .endBefore(position)©G. Malnati, J. Xhembulla 2016-23 33
Using the MVVP pattern
• If an app is backed by Firestore, the MVVP pattern can be
largely simplified
o There is no need to create a Repository with the corresponding Model
and RemoteDataSource, since Firestore provides its own local cache
and support offline data persistence
Activty / Fragment
ViewModel LiveData LiveData
LiveData
Firestore©G. Malnati, J. Xhembulla 2016-23 34
Using the MVVP pattern
• LiveData objects inside the viewModel can be populated by
subscribing to a query snapshot at creation time
o And removing the subscription when the viewModel is cleared
class MainViewModel: ViewModel() {
private val _courses = MutableLiveData<List<Course>>()
val courses: LiveData<List<Course>> = _courses
private val l: ListenerRegistration
init {
l = FirebaseFirestore.getInstance().collection(“courses”)
.addSnapshotListener{ r,e ->
_courses.value = if (e!=null)
emptyList()
else r.mapNotNull { d -> d.toCourse() }
}
override fun onCleared() { super.onCleared(); l.remove(); }
}©G. Malnati, J. Xhembulla 2016-23 35
Accessing data offline
• Cloud Firestore supports offline data persistence
o Data that is actively used is cached on device and is accessible when offline
o Read, write, listen and query operations are supported
o When the device is again online, local changes are synchronized with the
remote backend
• Offline persistence is enabled by default
o The FirebaseFirestoreSetting.Builder() class can be used to disable it or to
set the cache size©G. Malnati, J. Xhembulla 2016-23 36
Indexing data
• Query performance can be increased creating indices
o This is mandatory for making compound queries with ranges
• Indices are created and destroyed from the Firebase console or
form the Firebase command line interface (CLI)
o Indexes can take a few minutes to build, depending on the size of the
query and are subject to some limitations (200 max composite indices per
db and some other constraints)©G. Malnati, J. Xhembulla 2016-23 37
Concurrent modifications
• Multi-user applications must deal with concurrent data updates
(e.g., up-votes on a blog post)
o Firebase supports some ready-made operations (increment, arrayRemove,
arrayUnion)
o A transaction is a set of read and write operations on one or more
documents executed atomically
o A batched write is a set of write operations on one or more documents
executed atomically©G. Malnati, J. Xhembulla 2016-23 38
Transactions
• Transaction operations are described as a pure lambda function,
with no side-effects
o An optimistic locking approach can cause the function to be executed
several times, until all its pre-conditions are met
o Read operations must come before write ones
o Transaction will fail if the device is offline
o A value can be returned by the lambda to provide the transaction outcome
o Any exception thrown inside the lambda, causes the transaction to rollback©G. Malnati, J. Xhembulla 2016-23 39
Transactions
val sfDocRef = db.collection("cities").document("SF")
db.runTransaction { transaction ->
val snapshot = transaction.get(sfDocRef)
val newPop = snapshot.getDouble("population")!! + 1
if (newPop <= 1000000) {
transaction.update(sfDocRef, "population", newPop)
newPop
} else {
throw FirebaseFirestoreException("Too many people",
FirebaseFirestoreException.Code.ABORTED)
}
}.addOnSuccessListener { result ->
Log.d(TAG, "Transaction success: $result")
}.addOnFailureListener { e ->
Log.w(TAG, "Transaction failure.", e)
}©G. Malnati, J. Xhembulla 2016-23 40
Security : permissions
• By default Read/Write operations need the user to be
authenticated
o Rules are specified in a Domain Specific Language (DLS) in the Firebase
Console in the "Rules" tab
o It is possible to have different rules for different Collections/Objects
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
// This rule allows anyone on the internet to view, edit, and delete
// all data in your Firestore database. //
// Make sure to write security rules for your app before that time, or
// else your app will lose access to your Firestore database
match /{document=**} {
allow read, write: if request.time < timestamp.date(2020, 5, 18);
}
}
}©G. Malnati, J. Xhembulla 2016-23 41
Security : access rules
• Cloud Firestore Security Rules always begin with the following
declaration
service cloud.firestore {
match /databases/{database}/documents {
// ...
}
}©G. Malnati, J. Xhembulla 2016-23 42
Security: access rules
• Basic rules consist of a match statement specifying a document
path and an allow expression
service cloud.firestore {
match /databases/{database}/documents {
match /courses/{course} {
allow read: if <condition>;
allow write: if <condition>;
}
}
}©G. Malnati, J. Xhembulla 2016-23 43
Operation granularity
• A read clause can be split into get (a single document) and list (a
collection or a query)
o Write clauses can be split into create (a non-existing document), update
(an existing document) and delete
service cloud.firestore {
match /databases/{database}/documents {
match /courses/{course} {
allow get: if <condition>;
allow list: if <condition>;
}
match /courses/{course} {
allow create: if <condition>;
allow update: if <condition>;
allow delete: if <condition>;
}
}
}©G. Malnati, J. Xhembulla 2016-23 44
Expressing security conditions
• Access always
o match /collection/{id} {
allow read, write: if true;
}
• Authenticated users only
o match /collection/{id} {
allow write: if request.auth.uid != null;
}
• Users can only see their own data
o match /users/{userId} {
allow read, update, delete:
if request.auth.uid == userId;
allow create: if request.auth.uid !=null;
}