Our micro-front-end web architecture

Preface

Currently, I am the Front-End Engineer on our development team. My job, outside of developing much of the front-end portion of our new product development, is to focus on overall Front-End architecture and the shared reusable components of the code which can be easily implemented by Back-End/Full-Stack developers who may not be interested in the front-end. In the current state of things, these BE(Back-End) Devs can pull off quite a bit on the front-end side of things without being super familiar with the underlying frameworks and technologies involved, by using the cookie-cutter templates (see below) and libraries I've helped to create. Very rarely do they even have to create new CSS styles to spin up entirely new apps and generate the UIs for new products and features they're working on.

While I am the only official Front-End Engineer at Aver, I certainly did not design and build this all on my own. I have the pleasure of working with many brilliant full-stack developers who both coded and provided insights that helped shape this architecture. The architecture detailed below is one which was started ~2.5 years ago and has grown and morphed significantly over that period.

The current state of things:

We have over 22 distinct AngularJS Single Page Web Applications which make up a single product/site.
These applications are small, loosely coupled, designed for a single purpose and are easy to scale.
Each App lives under a top-level URL path such as
localhost/App1/
localhost/App2/
localhost/App2/dashboard
One app, home-web, owns the absolute top-level localhost/
Each app has its own deployment cycle
Typically each one of these front-end apps is accompanied by 1 or more back-end APIs who serve data to the app but any FE app is capable of utilizing any back-end api. The user-management-api for instance, is used by many different fe applications.
Navigating between apps executes a full-page reload but navigating between pages in a single app does not.
Navigation between the apps happens often and seamlessly.
The apps all share a single CSS Library so the look and feel across all of the apps is cohesive and navigating between them feels as if it were one monolithic application. This library has it's own Repo and NPM private registration. When the style library gets updated, all of our apps need to be rebuilt to pick up the updated styles or they'll be out of sync with one another and the transitions between them could potentially not feel succinct.
They share 90% the same code.
- The in-house shared code is pulled out into distinct AngularJS Modules/Components that have their own GitHub repo and NPM private registration.
- The 3rd party libraries are included via NPM and build into a vendor.bundle.js file using WebPack.
- The unique package.json 3rd party dependencies of any individual app are between 0-3, and these are only added when an app has a unique dependency that won't be used across the entire suite. If we find that every app is pulling in a particular library, we'll generally just make that a dependency of our ui-scaffolding library and pull it out of the individual app package.json files.
I've written a tool using CookieCutter which allows new front-end apps to be generated with via command line so that developers starting on a new app can start writing the code that gives the app purpose almost immediately without dealing with all of the typical boilerplate setup cruft. Here's what it does automagically:
- Prompts the user for any app-specific info such as:
  - App name
  - App base-url /user-management
  - Github repository info - It will automatically checkout the repo, generate the app into the folder, and make the initial commit of the app so you're ready to create a branch and start working immediately.
  - Includes demo assets - Example views, filters, directives, services and the accompanying unit tests to model after.
- Sets up all of the scaffolding which has to happen before you start writing the code you set out to write:
  - Angular app/module, Routing, Session logic, UI-Scaffolding.
  - Webpack Configuration
  - README.md
  - Package.json
  - ESLint Configuration
  - Unit test framework
  - Docker config
  - travis config
- Provides examples of Views, Components, Filters, and Services.
- Follows our code-standards and best practices.
- Integrates the app with our internal routing & dev-sandbox tools.
In Production/Test/E2E/deployed environments, for each app, we run a pre-built static version of the Application sitting in a docker-container running nginx.
In Development, for each app, we run the Webpack dev server in a container. On top of that, we run an in-house dev tool that sits in front of all of this to handle routing on localhost to the different applications as well as container management tools.

The good

Download only what you need:

Some of our Applications are what I would call "heavy" and have quite a bit of custom LESS, HTML, and JavaScript. Not all of our Users use, or even have access to, all of the other parts of the overall product. With our setup, the Users only download exactly the apps they are going to be using. If they don't hit a particular part of the application, they won't download the source for it. This does mean that the first time a user hits an individual micro-fe-app that they will experience a slightly longer load time while that app's JS, CSS, HTML, and images are downloaded. Subsequent sub-pages hit on that app will use the cached sources.

Developers only check out the code they're working on

This goes the same way for Developers. When doing development, you can clone, update, and spin up the few docker containers and code that it takes to get your work done. So if you're working in the user-management-web app, you can spin up the docker container for just that, and then the supporting services such as home-web and your few APIs required to get authenticated. This is a huge improvement over having to have everything cloned locally and running at the same time which will get your computer fans really moving.

Deploy only what you need:

Any one of our Apps can be edited, tested, and deployed completely independent of the other Apps. This gives us a lot of flexibility in how we deploy code, it also keeps things very fast. Which leads me to..

Faster Builds

When FE Projects get large, they begin to take a long time to build. Even with a properly configured and updated Webpack, you're going to get slow build times as your app grows. By breaking our app up, we're only rebuilding the code that has changed. This also helps when running your local webpack dev-server, and getting live code rebuilds.

Breaking up ownership:

There are clear divisions of ownership and responsibility of code and products between developers. As our team grows, it is likely that our app's products could be owned by separate Development Teams. With our current setup that would be extremely easy to organize as there is a clear division of top-level products as well as components and libraries. For instance, we could have a single team assigned to the Bootstrap/CSS library who can make application-wide changes to the look and feel of the app without causing much trouble to other teams assigned to the different top-level applications. We could have a User Management team who only works in that one application and codebase. For the most part, these teams don’t need to know anything about each other.

Sunsetting an application is clean and easy

Sometimes it's decided to shut down or remove a part of the Application. With this setup, doing so is incredibly easy! Let's say we wanted to get rid of a part of the app localhost/ExperimentalApp. All we'd need to do is shut down that individual service and it's no longer accessible. We'd then delete the Repo, remove any broken links in any other apps (I should figure out a better way for this), and it's gone! We're not left with old CSS, Angular Components, views, routing logic or anything else that might linger after the pages/apps are gone. I've worked on some pretty large, long-running, web-applications at prior jobs and I can tell you that the amount of lingering CSS is kind of terrifying. Generally, the styles are so potentially coupled with other pages or areas of the site that people are afraid to delete it. So what you end up with is thousands of style definitions or JS functions/components that are no longer being used; but they live on forever, getting bundled up and downloaded every time a user hits your site.

The Bad

Keeping apps in sync can be a challenge

Let's say we update our UI library and want to propagate those changes out to all of our FE Applications. Every FE App will need individual rebuilt & redeployed. But it gets worse. If we pin the UI Library to a specific version in our package.json file for each app, then we have to manually go through every FE application and update the version, open an PR, get someone to review it, merge it into master, rebuild & redeploy. This isn't so bad when you 3 Micro-services, but as I said above, we have more than 25. This gets exhausting! Especially when you get through all of this once, and then realize you need to immediately update it for something else. If anyone knows a solution for this please let me know!

We could just not pin the version (ui-library: "*") or only pin to the next major version (ui-library: "^3.2.1") but then that makes other things a bit more complicated, like testing. For instance, if someone pushes a new shared library update, any app that gets built-deployed from that point forward, who doesn't have the version pinned, is going to pull in the updated library. This isn't always clear that it's happening. If things break, it could be totally unrelated to the app code changes which initiated the deployment and have the developer scratching their head and sending them on a wild goose chase. This is no bueno!

I'm actively looking for a solution to this problem so if anyone has any ideas let me know!

Docker isn't great for webpack containers with file watching

One thing that we run into is the overhead of running many docker containers for the different front-end applications. Locally, each one is not running a static compiled version of the Application. It's running a full-on webpack dev server with file-watching and live reload. In fact here is the line we run webpack-dev-server --host 0.0.0.0 --port 80 --progress --colors --inline --hot. I think there are some definite optimizations that can be made to make sure that we're not running the live-reload file-watching over the node-modules folder or other parts of the code that don't require it. I've made some of these optimizations but I think there's room for improvement yet.

Typically, if you run the dev tool that's basically "Fire up all of the things" locally, your macbook fan will kick on full blast and sound like a harrier jet for 10-20 minutes. Even after everything is started and is sitting idle, it's not uncommon that your computer runs hot with it's fan on max speed. One thing we've done to combat this problem is being able to mark either a FE or API container as "Static". When done on front-end containers, the tool uses the static pre-built container for that app and skips running the webpack dev server or file watching. The end result is a lightweight instance of that Application, with the caveat that you're unable to make changes in the code and have them be reflected in the page.

If anyone has tips on running many webpack dev servers in docker containers locally in a more efficient way I'd love to hear about it!

How it works

There are a few different strategies for how you bring all of your Micro-FEs together. I've seen where you have a home-web that imports all of the other apps and it's deployed as a single app, but that's not how we do it.

Development

Locally all of the apps run in their own webpack-dev-server inside of their own docker-container. And the local code is mounted into the container. That way when you make changes, the changes appear in the docker container, webpack-dev-server sees them and recompiles the app, and the browser page refreshes. We use an internal tool locally for routing all of the docker containers to localhost.

Deployment

We actually deploy each app as individual docker images with Amazon Elastic Container Service.
Here's how it works:

Code gets committed to master, merged from a PR
TravisCI runs a series of tests including unit tests and other things.
Travis runs webpack build to create the build artifacts such as the dist directory of our FE app.
That dist directory is copied into an NginX docker-container and that container is pushed to Amazon Elastic Container Service.
Routing is configured via EC2 to point a domain such as www.myawesomesite.com/App1 to that docker container running on Amazon ECS.

So what's next?

The shared FE code is still downloaded, once-per app

Right now, the generated/bundled vendor.bundle.js?hash file which is deployed for each FE app is 100% identical to the vendor.bundle.js file for any other FE app. This giant chunk of JS, which includes AngularJS, Angular UI Router, and many other fairly large libraries are downloaded over and over again as you navigate between our Apps. I know this is a totally solvable problem through proper Webpack configuration & deployment setup but I've yet to hunker down and work out a solution. If anyone has any tips on how to solve this problem please share.

We're tightly couple to AngularJS

I think in a more ideal world our shared libraries and components are not dependent on AngularJS and rather just plain ol vanilla JS. Right now every piece of shared JS is an AngularJS 1.6.1 component/filter/service/etc. This breaks the paradigm that our micro-services are technology/library agnostic. If we genericized these libraries we could have a team build a new micro-fe-app in whatever FE framework they wanted, such as Angular2, React, or Vue while still utilizing that shared code. Some solutions to this problem include:

We could work towards re-writing the scaffolding code and shared libraries in a framework-agnostic format so anyone could use it, like plain ol vanilla JS. This seems simple enough for services but I'm not quite sure how to handle templates where you have library-specific portions such as ng-repeat or ng-if.
We could use a tool like SingleSpa which allows you to run multiple different front-end frameworks in a single page. I haven't spent time looking into this further but I'll need to very soon.
I could leave all the shared code as Angular components and use tools like Angular2React and ngImport to include those components right into my ReactApps. The major downside of this approach is that it will require new specialized tooling for every receiving library. For instance, if someone chose to create a Vue front-end, Angular2React won't be much help.
I could use Predix to re-write all of the components as predix components which are supposed to be framework-agnostic.

If you're familiar with other solutions this problem, or ways of migrating yourself out of it, I'd love to hear!

What I'm doing now

Well, I've begun prototyping and experimenting with moving from AngularJS@1.6 to ReactJS. By that, I don't mean migrating all of our front-end-micro-services over, but rather coming up with a strategy where our legacy AngularJS apps live side-by-side with the newer ReactJS apps. I'll be covering that process in a WIP follow-up post that you can find here: http://coreysnyder.me/it-has-begun-moving-from-angular-1-6-to-reactjs/