The microservices architectural pattern shines in use cases where scaling and deployment related problems need to be addressed by large tech teams. It is easy for organizations having several services to end up needing fleets of hundreds of servers to host them, without the need to support any real scalability problems. This often leads to wasted hardware resources as well as operational overheads managing so many deployments. In small to medium scale, this additional cost burden might never outweigh the benefits that they stand to gain by adopting the microservices pattern.

Keeping aside the cons, the advantages that microservices bring to teams of all sizes remain invaluable, often pushing teams to wrongly decide in favor of microservices. Each service having its own roadmap and a dedicated team removes all the bureaucratic processes that are inevitable with large monoliths. Also, the teams not needing to step on each other's toes, or understanding the complete codebase of a monolith, greatly improves productivity. Finally, the independence of deployment greatly enhances the release cycle, not having to queue up with the release plan of ‘other features’ in a monolith. 

Zango is an open source web application development framework that is designed to build multiple apps or microservices as tenants on a single monolith under the hood. Each microservice can have its isolated data models, separate codebase and even independent deployment. Errors in one  will not impact any other  and releasing updates in one microservice or app will not need the server to be restarted. We have open sourced the framework  recently but  have been using it to run 100s of production grade microservices on a single deployment successfully.  

At the core, Zango utilizes django-tenants library to create the tenancy model using Postgres’ schemas. With this approach every tenant gets its own space in the database that can be used to store its own table. In a request-response cycle, the tenant gets identified with its unique domain_url and the schema_path is set to the dedicated schema of the tenant. 

However, there is a problem here. django-tenants assumes that each tenant will be an exact replica of the other in terms of codebase (and thereby the models), and all the tenants get deployed together.  For each tenant to function as full fledged microservice, a few fundamental requirements are:

Let’s look at each of these challenges in detail and how we overcame them.  

Tenants as a plugins on our Django monolith

As our tenant represents an independent microservice, we must be able to support completely independent codebases for each tenant. Since a given project can have 100s of tenants, we cannot allow the code to be compiled together in the usual Django way, otherwise with each new tenant added the codebase keeps swelling, handling migration etc becomes more difficult, etc. We need a new pattern altogether to support our requirement. 

After trying a number of different approaches, what worked for us was to use the plugin architectural pattern. Each tenant can be considered as a plugin on our monolith. Yes, this approach would separate our tenant  from the Django system in a major way, impacting routing, models, migrations and almost everything. But we will look at how we handled these things shortly.

For implementing the plugin system, we used pluginbase, a small python module published by Armin Ronacher, the creator of flask, for development of flexible plugin systems in python.

It allows loading plugins in the below simple steps:

from pluginbase import PluginBase
plugin_base = PluginBase(package='yourapplication.plugins')
plugin_source = plugin_base.make_plugin_source(
           searchpath=['./path/to/plugins', './path/to/more/plugins'])
my_plugin = plugin_source.load_plugin('my_plugin')
My_plugin.do_something_cool()

Make the ORM work bypassing Django’s apps registry

Now with the handling of a tenant's codebase done, another challenge before us was the integration with Django ORM. Our plugin based tenants should have the flexibility to define their own models, be able to use the migrations system and use ORM features like normal Django apps. It so happens that Django’s ORM is tightly coupled with the apps registry and other Django internals that are not so easy to hack into. 

After spending countless hours studying the implementation of Django ORM, apps  registry, and other public resources, we were finally able to work out a way for the ORM to work bypassing the apps. We created an app at our framework level called dynamic_models and created a abstract base model here:

class DynamicModelBase(models.Model,  metaclass=RegisterOnceModelMeta):
    class Meta:
        app_label = "dynamic_models"
        apps = DefaultAppsProxy()
        abstract = True

The RegisterOnceModelMeta ensures our dynamic models will be registered only once and not every time the plugin is loaded. Also, the DefaultAppsProxy is a proxy to the default apps registry. It is needed to make our dynamic models available in the options when the relation tree is built.

Finally, the AppConfig for the dynamic_model app has to override the ready method to do the pending operations 

class DynamicModelsConfig(AppConfig):
   default_auto_field = 'django.db.models.BigAutoField'
   name = 'zelthy.apps.dynamic_models'

   def ready(self):
       # thanks to Baserow for this hack
       import zelthy.apps.dynamic_models.signals
       original_register_model = self.apps.register_model

       def register_model(app_label, model):
           if 'ws_makemigration' in sys.argv:
               original_register_model(app_label, model)
           else:
               pre_save.connect(set_created_modified_by, sender=model)
               self.apps.do_pending_operations(model)
               self.apps.clear_cache()
       self.apps.register_model = register_model

Tenants can have overlapping URI’s but distinct implementation

Another major fundamental deviation from the core Django way of doing things would be the way routing will work in our framework. Tenants can have their own views and urls and since they are loaded as plugins and also have overlapping URI’s, we cannot work with Django's default routing system. So we create our own.

Our framework requires tenants to specify the root urls in a specific file (settings.json) located in the root of the tenant’s codebase. We define a open route at the framework level which will point to an EntryPointView.

urlpatterns = [
    re_path(
        r'^((?:[\w\-:.,]+/)*)$', 
        EntryPointView.as_view(),
        name='dynamic-app'
    )
]

The EntryPointView honors all incoming requests and being already aware about the specific tenant  for which the request is intended, it attempts to find the matching route in the tenant. If the route is found, the attached view is loaded through the plugin system and served as the response, otherwise the EntryPointView returns Http404.

class EntryPointView(View, PermMixin):
    """
    this class is responsible for building the
    """
    def dispatch(self, request, *args, **kwargs):
        if self.has_user_access_perm(request, *args, **kwargs):
            self.workspace = self.get_workspace(request)
            view, resolve = self.get_view(request)
            view_name = (
                ".".join(resolve.__dict__["_func_path"].split(".")[3:])
                if resolve else None
            )
            if view and view_name:
                kwargs = resolve.__dict__["kwargs"]
                if request.internal_routing or self.has_view_perm(
                    request, view_name, *args, **kwargs
                ):
                    response = csrf_protect(view)(request, *args, **kwargs)
                    return response
            )
            if view and view_name:
                kwargs = resolve.__dict__["kwargs"]
                if request.internal_routing or self.has_view_perm(
                    request, view_name, *args, **kwargs
                ):
                    response = csrf_protect(view)(request, *args, **kwargs)
                    return response

With the multi-tenant system that we have created so far, where tenants can be added on the fly as plugins, it is supremely important to have hard boundaries. The framework must provide for a strong policy implementation to ensure no unprivileged access happens, even between tenants. For this we have implemented a policy framework that requires policy statements (as JSON) to be created and mapped with users and user roles for accessing any resource (e.g. views, ORM queries, etc.).

Finally, the internal routing system between our microservices

Since the framework is supposed to function as a microservice as a tenant, we can expect requests originating from one tenant to another. We definitely do not want these requests to go to the public network and come back to be served from an adjacent tenant. To prevent this, we have provisioned for an internal routing between tenants. For this we have monkey patched the requests library that looks for the destination domain_url and if it is one of the sister tenants, then it serves internally. Of course the policy system ensures that this pipeline is well governed.

large_Deep Dive

medium_Deep Dive

small_Deep Dive

thumbnail_Deep Dive

Deep Dive

Deep dive into Zango's architecture

You must be this tall to use Microservices - Martin Fowler

Microservices architectural pattern has gained massive prominence and widespread adoption over the last decade or so due to the many advantages it offers. It enables decentralized structures at the people level as well as at the technical level and improves the ability of systems to effectively respond  to the demands of growth, both of features and scale. The microservices pattern definitely offers higher elasticity to the system it is powering to respond to changes in a dynamic environment.

On the other hand, a monolithic system is akin to a  centralized structure where the complexity keeps increasing by the day. The addition of more lines of codes, new modules, new  packages, new dependencies, etc. leads to a system where the elasticity to change is decreasing. Enhancements tends to become more cumbersome and expensive and it is easier for more crufts to surface in the codebase. The burden of bureaucracy also tends to be higher on the monolith for obvious reasons.

large_developer_difficulty.png

medium_developer_difficulty.png

small_developer_difficulty.png

thumbnail_developer_difficulty.png

developer_difficulty.png

More recently, there has been a lot of discussion around the cost of microservices. The in-memory method calls in a monolith between modules becomes the more expensive network calls in microservices. Separate deployment makes it easy to run into a fleet of hundreds of servers in organizations running several microservices in production. Then there are overheads introduced from more complex monitoring, devops, etc.

While the ability to run and scale smaller independent systems introduces the leverage of agility, the higher infrastructural and operational overheads renders the choice of microservices impractical in a vast majority of use cases.

During the proliferation phase,  the microservices pattern seemed like the silver bullet and organizations of all sizes were adopting it overlooking the overheads. In the current times, it seems as if the reverse is happening. It is becoming fashionable to suggest that microservices is not a suitable solution for the problem at hand and that monolith is ‘majestic’ or start with a monolith. 

The advantages of microservices, albeit the cost

The microservices pattern can revolutionize the way a product is developed, deployed and enhanced even for ecosystems that might not have hit the threshold of scale. Keeping aside the cost and concerns around the operational overhead, microservices as a pattern stands to quickly outshine a monolithic approach by remaining more adaptable to new requirements. The enablement of parallel development by establishing well enforced boundaries greatly adds to the agility of the development team.  

The adage is to start with a monolith and branch out to microservices, using the approach known as the 

Don’t start with a monolith if the goal is microservices

I’m firmly convinced that starting with a monolith is usually exactly the wrong thing to do. Starting to build a new system is exactly the time when you should be thinking about carving it up into pieces. I strongly disagree with the idea that you can postpone this.

Building a greenfield monolith that is expected to grow out into a Strangler Fig and expecting to cut it off when the time is right will add stress to the design of the monolith in the first place. 

You might be tempted to assume there are a number of nicely separated microservices hiding in your monolith, just waiting to be extracted. In reality, though, it’s extremely hard to avoid creating lots of connections, planned and unplanned. In fact the whole point of the microservices approach is to make it hard to create something like this.

A new approach for microservices, sans the overheads

 is a new Python web application development framework, implemented on top of 

,  that allows leveraging the key advantages of the micorservices pattern without incurring any additional operational overheads.  

Under the hood, Zango is a monolith that allows hosting multiple independently deployable applications or microservices on top of it. Each application is logically isolated from the other sibling apps at the data layer, the logic layer, the interface layer as well as in deployment, ensuring complete microservices type autonomy. 

One of the key goals of Zango's design is to make sure that deployment of one app does not require  downtime for any other sibling app or the underlying monolith. We do so by creating logically separated areas, enforcing rules and policies at the framework level and leveraging the plugin architectural pattern for executing the app's logics. 

In addition, to meet the requirement of differential scaling, Zango also allows easy branching off of one or more services to deployment on a separate cluster without needing any code level refactoring. 

Zango has been open sourced recently, but it has been in production to support hundreds of applications in global companies. 

Let’s understand the architecture in detail.

Zango comes with Postgres SQL database and the organization of the datastore is central to the framework.  We utilize the 

feature of Postgres to logically segregate the storage for each microservice. Each microservice will have a unique schema in the database where all the tables and associated objects for that microservice will be housed. The image below shows the organization at the database level.

large_db_schema.png

medium_db_schema.png

small_db_schema.png

thumbnail_db_schema.png

db_schema.png

Policies are enforced by the framework to prevent unauthorized access to data. For e.g. application logic implemented in microservice 1 that attempts to access the data for microservice 2 will be restricted by the default policies.

Additionally the  common schema in the database will have a table to function as the registry of microservices. In this registry we store the generic details of the microservices, including:

With the organization of data storage for the microservices in place, we can now look into how Zango handles the codebase, ensuring we meet the key requirements:

To meet these requirements, we leverage the Plugin Architectural pattern. Each microservice is implemented as a plugin. Caching techniques ensure the efficient loading of the plugin codebase and re-loading only when new releases are available for a particular microservice.

On receiving an incoming request, Zango first attempts to identify which microservice it is intended for, using the domain url from the request and the matching with the ones stored in the registry. If a matching microservice is found, an attempt to find a matching route for the URI in the microservice’s codebase. The framework enforces certain rules for defining the routes. If a matching route is found, the controller associated with the route is called to generate the response. If the match is not found, 404 is returned. 

The framework also sets the path to the particular schema in the database that belongs to the microservice. By doing so, there will be no requirement to explicitly set the schema context in the  codebase of the microservice and the developer gets the sense that he is working with a single tenant database only. 

The overall architecture involves a lot of other nuances to handle finer details around access control and authorization framework, differential throttling, packages layer to enable a packages ecosystem, etc. which is beyond the scope of this introduction. 

We believe that the possibility of implementing a microservices architecture without additional cost burden can bring a lot of advantages to a greenfield web application project. The power of differential scaling on the same deployment as well as the ease with which the microservices can be extracted fully into an independent deployment will significantly strengthen the teams

Zango is available under an  open source AGPL license 

. It is implemented on top of Django and requires basic Django understanding to start developing applications on it.

Deep dive into Zango's architecture

Architecting Microservices as Tenants on a Monolith