Why Firestore, Part III:  6 Things You Need to Know Before Using Firestore

With the rise of cross-platform technologies like Flutter, React Native, and Kotlin Multiplatform, Firestore is gaining more and more attention as it tries to solve quite a few problems with the platform. Not only does it promise to replace fully-fledged backend solutions, but it also allows you to write mobile code only. But is this the silver bullet that developers are trying to find?

The last time we looked at the Firestore (and Firebase in general), we saw some of its problems. These don’t make Cloud Firestore a bad technology, no. It is still one of the easiest real-time databases to use for Android, iOS, desktop, and web. Considering all the limitations, now is the time to answer the natural follow-up question - how to overcome the Firestore problems?

The short answer would be, "don’t use it,” but let’s be honest here. Firestore can be a timesaver for some problems, and it might be worth using it, despite all the problems. Just not overuse it and be ready to change it to something more appropriate later.

In this post, I want to give you some hints on how to use Firestore effectively and not hurt yourself in the process. Most of the time, this doesn’t involve code at all - you just need to design your app appropriately :) Read on to find out how!

This is the final article from a series where we comprehensively describe the pros and cons of using Firestore as the backend solution and database for your next mobile or web application. In this series, we will try to show you that making this decision is not a simple process, and you need to analyze your app from multiple perspectives.

Posts in the series:

1. Why Firestore, Part I: reasons to love it.
2. Why Firestore, Part II: reasons to hate it.
3. Why Firestore, Part III: 6 things you need to know before using Firestore [you are currently reading it].
4. Why Firestore, Part IV: how to escape it.

Would you like to learn from our past experience what can go wrong with Firestore and Firebase and what are their biggest disadvantages and limitations? Read on to find out why!

Embrace it

In the previous post, I described the pain points of Firestore. The list is incomplete but gives you an idea of what to expect and what roadblocks you will have to overcome. To sum things up, Firestore lacks in terms of:

1. Security,
2. Costs,
3. Latency,
4. DB features,
5. Data modeling possibilities,
6. Backups.

All problems can be mitigated when you acknowledge their existence. This is the main thing to remember when using Firestore. You have to embrace them and try to overcome them this way or another. You won’t solve them (at least not now), but you can keep them in check and maybe use them to your advantage.

In this section, I will try to describe ways to handle all the drawbacks of the previous post. This is nowhere near complete or full, but hopefully, it will give you a starting point.

1. Isolate and secure sensitive data

You will have holes in your Firestore rules sooner or later if you need to do anything more complicated. This is inevitable, no matter how hard you try. They are just too complex. There is a possibility that nothing bad will happen because you fix them just in time, but when it comes to really sensitive data, it’s better not to leave any possibility of a mistake.

My recommendation would be to isolate the sensitive data and put it elsewhere. Let them stay in an ordinary, backend-only database and guard everything with proper, API-level authorization. It is much easier to secure, and you have the means to test it and audit it automatically (which is not possible or very cumbersome with Firestore). You can put everything else in the Firestore, but the most precious data will be kept secure.

If you really don’t want to have separate storage, at least isolate it on the Firestore level (e.g., separate collection) and put much more scrutiny on the rules guarding this part of the DB. It will require more work, but at least your data won’t leak.

2. Design your data for the pricing model

As I said in the previous post, Firestore pricing is hard. You really have to think about it from the get-go. Otherwise, you might end up with a Very High Bill at the end of the month. From my experience, the main money hog is the indexing engine. If you leave the defaults for every document, you will generate multiples of the document size in indices, and Firestore storage is pricey.

My advice? Limit the data you put there (i.e., prevent over-duplication). Add exemptions for fields you don’t use in searches. Don’t use dynamic field names. Don’t store binary data there - use GCS/S3/Azure Blobs to link it. Remove old or unused data regularly. Limit the amount of user-generated content there (or sanitize it beforehand). Don’t put long strings there - treat these the same as binary data. And the simplest yet often overlooked advice - design your data well.

3. Workaround latency

Unfortunately, there is nothing you can do to solve this. Until Google gives stronger guarantees, we can’t promise anything to our users.

But not all hope is lost. There is always a way to hide the latency, either by moving the work to the background (and, e.g., preloading the data) or using some clever tricks in the UI (spinner being the easiest one :) ). You still can’t give your users any guarantees and need to prepare them for sporadic lags, but nothing will break if you expect it to happen.

If you have a very time-sensitive app, like online auctions, where you can’t have unpredictable delays (no one likes to lose because of something that we don’t control), you shouldn’t really use Firestore. You will have a hard time hiding the hiccups, and you will only anger your users. There are more places where Firestore doesn’t fit, but for that, we have another section below.

4. Don’t put too much on it

Firestore is not designed to work with great amounts of data at once. It also doesn’t have the most sophisticated query engine. That said, it does work quite well, and you can do some simple searches with it to get the job done.

Sometimes you have to design your data so that you can express the query (because of the limitations) at all. You might need to denormalize and duplicate the data to facilitate the query or use some strange encoding so that it is expressible (e.g., put a couple of fields into one string and search on that :) ). You must be prepared to sacrifice some functionality or go with a more involved solution. Nevertheless, with some up-front design, you can make it work the way you want.

5. If you select it, do it the right way

If you want to use Firestore as your main database and skip the backend altogether, you really don’t want to go the easy way here. The NoSQL approach, especially client-side NoSQL, with Firestore’s limitations, requires you to think over the design upfront. Really think it over. You just can’t allow yourself to go with the flow and do (the bad version of) emergent design because it will bite you on the first possible occasion.

There are multiple approaches to the design. DDD is one that I would recommend personally. You can find great materials for it on the Internet, but I would recommend starting with Reso Coder’s Firebase & DDD tutorial in Flutter. It describes it well and also gives you well-modeled architecture that all your apps might use. Not only Flutter apps. All mobile (and desktop probably) apps, native Android/iOS, React Native, or even Xamarin.

With proper architecture and well-modeled data, you will be able to keep your app maintainable, and you might manage to overcome Firestore limitations with a scalability story at your disposal.

6. Configure periodic exports but be aware of the data loss

You can’t have proper backups, but you can leverage exports to slightly mitigate the issue. You still might lose some data and need to be prepared to solve consistency issues manually, but at least you will be able to restore it in case of failure. Google has an article on how to use Cloud Functions to automate tasks. It probably needs some tweaking (e.g., alerting when the export fails), but it is a great start.

And remember that Firestore has no backups.

Cloud Firestore doesn’t fit all use cases

Although Firestore solves many hard problems, it doesn’t fit everything. There are cases when using Firestore will give you negligible gain compared to the problems that it will induce. You have to carefully weigh the pros and cons and decide for yourself, but here are a couple of cases that I think make it a substandard choice.

Proper backend

If you’re building a project with a fully-fledged backend in Python, Ruby, .NET, Go, Rust, or any other technology (COBOL, anyone? :) ), it will be easier to use it. Although Firestore might seem simpler, if you put too much there (and it will be tempting), you will quickly end up with convoluted architecture. Especially if you keep the backend code. Then you will have different data in different places, and your data layer will need to be very sophisticated.

Firestore might be a viable option for a small part of the app where you would really use its features. You will just need to maintain the programming regime and don’t succumb to the “yay, let’s use it for everything” vibe. :)

Security requirements

When it comes to PII, financial or medical data, you can’t really take shortcuts here. You can put PII in Cloud Firestore, but I don’t think anything more sensitive should go there. Securing this particular database is hard; in my opinion, you shouldn’t compromise here. There is too much at stake to allow any mistake.

I don’t even think that keeping PII, there should be done. I’ve seen projects that used Firebase Authentication and kept user profiles in Firestore (in addition to Auth), which isn’t inherently a bad idea. Except when you have no security rules, or these are broken. Without proper authorization, this data is effectively public, and you have a data leak from day one.

Latency-sensitive apps

Let’s say you’re building an app for live auctions. You’re considering using Cloud Firestore as your backend for auctions, where you plan to keep everything - both descriptive info, prices, and live bids. This might seem like a good idea. Using the reactive approach with Firestore (subscriptions) will allow you to quickly build the refresh mechanism forbids that is perceivably instant.

Except when it won’t. You don’t control how Firestore libraries behave regarding caching or connectivity. You don’t control when they sync the data. You can’t force that. This is also an implementation detail, so Google doesn’t really describe how it works under the hood. If you have bad luck, one of your users might totally think that they won the auction because, in the last second, Firestore hasn’t sent an update.

Complex apps

Nothing beats separation of concerns. Having everything client-side will be very hard to maintain. You also limit the team size that can work on the app at once because they will get in one's way. If your app gets near the complex phase, Firestore will bite you and substantially slow development. You can’t really develop an app indefinitely, having everything in the same place. This will be just too hard and too error-prone.

You will probably also stumble upon missing features. Firestore has a query engine, but it won’t replace ElasticSearch. It has subscriptions, but these are not a substitute for WebSocket-based communication. You can attach functions to events, but it won’t replace other solutions (e.g., Airflow) when it comes to batch processing. It doesn’t even aim to do so; you need to be aware of that.

Summary

As you can see from the first article in our series, Firestore (and Firebase!) might be a great technology if used appropriately. It allows you to build some features really fast and deliver functionality in no time. It is very pleasant to work with, especially in mobile technologies, and integrates well with the ecosystem. With the offline capabilities at hand, you can build some great functionality at a minimal cost.

It also has its fair share of drawbacks. Designing the data to fit Firestore's model might be quite hard. You will have to address some of its limitations upfront (latency, migration possibility), keep enough scrutiny to not break things (security rules), and just get used to some things (pricing).

In this article, I tried to show you how you can address some of the limitations. I also described types of apps that, in my opinion, won’t benefit from Firestore. The features of Firestore are indeed compelling, but with all the problems with it, it just might not be worth it. You need to weigh the pros and cons and make informed decisions regarding Firestore because once selected, you might not be able to easily go back and choose something different.

But not all hope is lost. Even if you use Firestore already but want to escape it, there are ways to do so. This, and probably more, will be part of the next article, unexpected even for us, but we think it is necessary. Follow us on Twitter or other social media to avoid missing it!