• Hooks + useReducer + useContext = Mini Redux with Hooks

This article is a WIP and likely contains plenty of spelling and grammatical errors, as well as many incomplete parts. Hopefully, I'll have time to continue updating it.

Right from the start, there are 2 things that you need to buy, which you're going to use for all of your drones. I have around 30 drones and planes, and I use the same Goggles and Transmitter to fly all of them. So these are pretty key purchases. They can also be 2 of the most expensive things you purchase in the hobby but think of them as investments. Before we getting into suggested buying options, you need to understand a little bit about the frequencies involved.

Frequencies Involved

Beginner

Your drone is going to be using two different bands of frequencies simultaneously. One for video and one for controls.

• 2.4Ghz for your Controls
• 5.8Ghz for your Video

Controls: This uses 2.4Ghz. Your drone is connected to, typically, a 2-way communication system. your Transmitter (AKA TX or Controller), is going to be sending controls signals to your drone. When you press left, it sends a signal to your drone to go left. Your drone can also send data back to your Transmitter on this same signal, that's why I say 2-way. Your drone can tell your controller things like the drone's battery voltage, or what video frequency you're on. But that's getting too advanced for this article.

Video: This uses 5.8Ghz. Your drone will have a Video Transmitter (AKA VTX) which will allow your drone to send the video of what it sees through its camera, wirelessly, to any receiver who's dialed to the same frequency. This means that anyone with a receiver capable of receiving your frequency can watch your video.

Advanced

These two frequencies will cover most of your bases but if you start getting into more advanced activities like Long Range Flying or HD Video you may need to check out the following:

Advanced Controls

• TBS Crossfire which uses the following frequencies: 868MHz (EU, Russia) / 915MHz (USA, Asia, Australia)

Advanced Video Frequencies

Your goggles may have a built-in receiving module, but there are options that exist that can give you a better video signal.

This may come in the form of Googgle-bay mods such as:

• FuriousFPV TrueD Module
• Foxeer Wildfire
• ImmersionRC RapidFIRE
• Iftron ClearView Goggle Receiver Module

HD Video Technology

• DJI Digital FPV System
• Connex Digital FPV System. I'd personally stay away from this one unless you know what it is, and specifically want to use it for its features. For noobies, it's just going to cost you a lot of money and frustration.

Next, let's talk about what you'll need to send/receive those signals.

Goggles / Monitor

Beginner

If you want to see what your drone sees, you'll need to have something set up to receive that video. For most of the FPV Racing industry that's going to be a pair of goggles. You can also use a monitor or hooked up to a Video Receiver though as well. This is great for sharing your video feed with other people so they can "fly along".

There are two main types of goggles:

1. Box Goggles - Box goggles are going to be the cheapest entry point typically. They're bulkier, and typically have a single, lower-resolution screen, but still, give you that sense of FPV immersion. These are great starter goggles but you're likely going to find yourself eventually wanting one of the goggles below.

Some great options for box goggles include:

• Fat Shark Recon
• Topsky Prime 1S FPV

2. Form-Fitting goggles - These are usually more expensive, smaller, and have two, higher definition, screens inside; One for each eye.

Some great options for form-fitting goggles include:

• Fat Shark Dominator HDO
• Fat Shark Dominator HD3

Advanced

You can get into some advanced setups for doing things like putting your video feed on a big screen with others side-by-side. This is great for getting spectators in on the action.

Another more advanced topic is HD Digital video. If you're interested in that check out the DJI or Connex systems listed above.

Transmitter / Controller / TX

This is where you have to make a decision that will follow you around for a while. When it comes to the 2.4Ghz controller frequencies, there are several different protocols, and they typically don't all work together. Spektrum, for instance, is a company. If you buy the Spektrum transmitter, you can only use it with receivers that are set up to use the Spektrum protocol.

The two I'm most familiar with and have used are:

• Spektrum
• FrSky

I would nudge new pilots toward FrSky. Spektrum is typically more expensive, moves a little slower in terms of Drone tech, and I've had issues with mine in the past. So you need to make a decision right now which you're going to use. Order your transmitter and stick with it.

Get the best of both worlds! There are also Transmitters such as the Jumper T16 Pro or the RadioMaster T16 which can connect to most of the protocols out there. These are both high value / low cost radios for those looking to get into the hobby though they do require a little more work out of the box than say Binding a Spektrum Radio to a BNF Quad. If you're trying to get started on the cheap, look @ for an older model that uses the protocol you're interested in.

The parts of the drones

Building a drone can be a lot like building a PC. There are a bunch of different parts that each serve a unique purpose. But it can be confusing to know what you need, and what parts work with what. If you're not sure what to buy, check out RotoRage. It will allow you to piece together a drone and give you alerts if you try to add parts that are not compatible with one another.

You should do some research to make sure you're parts are compatible but in general, here are the parts you'll need.

At a high level here are the parts you'll need:

• Frame
• Flight Controller (FC)
• Battery
• Motors
• ESCs
• Camera
• VTX
• Antenna
• Propellers

Frame

This is what all of your parts connect to. This selection will limit exactly what you can and cannot use. Frames are typically designed to carry a certain size flight-controller stack, motor sizes, camera sizes, etc. Selecting this first will help limit your potential options for the following. For help deciding on which frame to buy, read the Types of FPV Drones below.

Flight Controller (FC)

This is the brain of the operation which keeps the drone in the air and properly responding to your controls inputs.

The considerations for choosing a flight controller include:

• Size: Most Flight Controllers come in 2 sizes for mounting holes, 30.5x30.5mm and 20x20mm.
• Software: Which software you want to run on it such as Betaflight, Butterflight, Raceflight, KISS.
• Features: What features you need such as # of UARTs.

Battery

This going to power your drone. In the industry, we typically use Lithium Polymer(LiPo) batteries. It's especially important to verify that your parts can take the voltage of the battery you're trying to use. This is the #1 fastest way to ruin your drone. Batteries are labeled by 2 main attributes. The # of cells they include, and the capacity.

Examples:

• 3S 800mah - This is a 3-Cell 800 milliamp battery
• 4S 1300mah - This is a 4-cell 1300 milliamp battery

So why does MAH matter?
So the higher the MAH of your battery, the longer you can fly. But it comes at the cost of weight. You'll need to decide what weight you're after. Here are some example sizes I use:
400-800mah - Small 2-3" drones
1100-1550mah - 5" Race Drones
1300-1800mah - 5" Freestyle drones
1800-2500mah - 5 & 6" Long-Range drones.

Carrying a larger battery will not always get you a longer flight time b/c of weight. There is a point of diminishing returns and you'll have to find that for your drone if you stray from my general guidelines above.

So why does voltage matter
You can read more about this in the motors section but essentially the higher the voltage, the faster your motors spin. But faster isn't always better, and it can damage your drone.

The new rage right now is using higher voltage batteries like 6S. It's generally accepted that they help with voltage sag and increase race longevity. If you're new to the hobby and want to keep costs low, I'd recommend going 4S. You may be able to buy batteries of someone whos trying to move to 6S. In my opinion, if you're jsut getting started out, you're not going to notice the difference between a 4S and 6S quad except the cost of parts.

C-Rating:
You may also see a third notation on the battery like 75C. That's the C Rating of the battery and is an indicator of the continuous discharge rate of a LiPo. It allows you to calculate the maximum constant current you can draw from the LiPo pack safely without harming the battery. For drones don't buy a battery with a C rating of under 45. And the higher the better for maintaining fast forward flight or big punchouts.

Proper Battery Care

If you don't take care of your batteries they will quickly become junk. The fastest way to take a $60 battery and ruin it is to plug it into your drone, and fly for 10 minutes or until the drone comes due to there being no more juice. If you do this, you've now destroyed your battery. You may notice it's hot to the touch, swollen, and could burst into flames at any moment depending on how quickly you discharged it. If you've done this, throw your battery off to the side away from your stuff and let it rest a while.

If you do this there are a few outcomes for your battery:

1. If you let it sit, it may bounce back to a normal resting voltage and allow you to recharge and reuse it. Just know that it will never have the performance it once did.
2. It may be so low in voltage that you can no longer recharge it as your charger won't recognize it as a proper battery.
3. It may burst into flames and destroy your other drone components.

Another thing you should avoid doing is fully charging your batteries and then letting them sit for an extended period of time. When you're done flying you should try and fly all of your batteries down to 3.8v/cell or put them back on the charger and use the storage setting which will do the same, just a lot slower and less fun.

Motors

After you select your battery, you can select which motors to use. The motors are the muscles of your drone. They're what spin the props and allow you to whip that drone all over the sky.

Motors have a few numbers that tell you facts about the motor. A typical motor might say 2206 2300kv. That tells you that the motor is 22mm wide, 6mm tall, and spins at 2300kv. The reason the battery dictates the motor is because of that Kv spec. “Kv” refers to the constant velocity of a motor (not to be confused with “kV,” the abbreviation for kilovolt). It is measured by the number of revolutions per minute (rpm) that a motor turns when 1V (one volt) is applied with no load attached to that motor. So that means if you put a 4S battery on a motor it's going to spin at 4X that Kv. 6S battery would be 6X that Kv. Trying to spin a motor too fast, with too much load, can quickly fry it. There's kind of a balancing act between Battery, Motor, and Prop Size. You can use RotoRage.com to help you pick a motor that is good for the battery you're using.

A few examples:
DYS SE2205 PRO 2550KV - This is a 22mm tall, 5mm wide motor that spins at 2550kv. This is good for a 4S battery running 5" props, but could also work for a 3S battery.
BH Tornado 1407 3600kv - This is a 14mm wide, 7mm tall motor. This is going to be good for a 3" or 4" propeller, on 3S or 4S batteries.
Hyperlite 2207 1722kv - This is a 22mm wide, 7mm tall motor. This motor is designed for 5" race drones running 6S.

Electronic Speed Controllers (ESCs)

An ESC is the speed controller for your motor, so you'll need 1 for every motor. An industry standard has become the "4in1" which crams all 4 ESCs into a single stackable board. This is a good idea IMO, especially for new drone builders. It can make for really clean builds and keeps your ESCs out of the reach of your spinning propellers.

In my opinion, there's very little to decide about ESCs. You need to make sure you get one that is rated for the Voltage of battery you plan to use, and you want something that isn't cheap. A failed ESC sure sucks and it seems to be a common thing to go especially if you buy cheap.

Camera

This is the camera that will be your eyes in the sky. Don't confuse this with the GoPro that is on the drone. It's typically a pretty low-res, low-latency camera, not too different than the back-up camera in your car. When choosing a camera you only need to consider form-factor.

As you get more advanced you can look at cameras that have the color profile you like, are higher definition, or offer better low-light characteristics.

Video Transmitter (VTX)

The job of the VTX is to take a video feed from your camera and send it wirelessly outward through your transmitting antenna. When you're picking out a VTX, make sure you get one that is 5.8Ghz, the same as the goggles you bought above. When choosing a VTX, you want to choose one with the power output to match your application. Typically the higher the mW output, the longer the range. Here's a good rule of thumb:

• 25mW - Racing or indoor flying where you're staying close to yourself.
• 200mW - This is what we used to use for racing but it's less common.
• 500mW - Freestyle
• Over 500mw - Long Range flying

Receiver

The receiver's job is to communicate with your Transmitter/Controller/TX. It both receives control signals, and outputs telemetry data. When choosing a Receiver you'll need to pick one that is compatible with your Transmitter/Controller.

Antenna

This doesn't matter so much. Two things matter to me when selecting an antenna. #1, that it is built for the right frequency video I'm outputting. So make sure you get a 5.8Ghz antenna (most are).

And #2, form factor. For racing, I want one very small and snug against the frame so it doesn't get damaged or knocked off. Keeping it close to the other components will degrade video reception so that comes at a cost. For Long-Range or Freestyle when I want to be able to fly far out, around obstacles and need great video connection, I'll use a longer antenna that stays out away from the frame.

Propellers

The propellers go on your motors and are what spin around to create the lift for your drone. Propellers will dramatically affect your flight characteristics. They come in a variety of shapes and sizes. As such, they are very heavily personal preferences. You're going to have to fly a lot get and feel for what kind of props you like.

Types of FPV Drones

Ok so now that you've got your radio and goggles, you're ready to start that drone collection. FPV is a very addicting hobby and you'll find yourself with a fleet in no time.

When you are trying to pick out a drone it's important to consider what you're buying the drone for. Do you want to fly inside? In a small backyard? Do sweet freestyle flippy-floppies like Mr. Steele? Juicy Acro like JohnnyFPV? Fly long-range mountain surfing like Gab? Or to race like Nurk & Jet? those are all going to dictate what type of parts you order. You can kind of fly all of those with a single drone but I like to specialize.

So before you get started, think about what you're setting out to accomplish and let that drive some of your buying decisions.

I want to spend very little money and just want to experience FPV.

First, I'd say you should absolutely download a simulator and start practicing. Even before you fly a drone in real life. Check out the SIMULATOR section for more details.

NOTICE: Due to some of the recent legislation, it might be a good idea that your first drone weighs less than 250G AUW (Including the battery. Basically everything that flies).

When you're ready for your first real drone, Something like this is perfect!
If it's sold out, just go buy all the parts individually. It includes everything you need to experience FPV for yourself. That said, if you decide to get into it, you'll immediately want to upgrade your controller and probably goggles too. So if you want to save yourself having to waste that money maybe get the stand-alone kit and then buy a nicer set of goggles & controller.

While it is true that this will not fly 100% like a 5" race drone, it's going to the be the cheapest way to get started and those skills will transition to the 5" drone. Because it's so light it will break way less, and won't cause as much damage if you crash it into other things. Also, the parts are incredibly cheap so fixing it shouldn't cost you much.

Inside Flying

100% just buy a Tiny whoop as mentioned above.

Backyard or Large Indoor Flying

Build a 3" or smaller drone with brushless motors. These little rockets are plenty fast but are much more manageable in a tight area.

Outdoor Freestyle

Build a 5 or 6" quadcopter running 4-6S. Your primary focus for parts is going to be durability & part-protection. When you free-style you're inevitably going to crash. Freestyle drones are typically bulkier and offer your parts a lot more protection.

Racing Drones

Build a 5" quadcopter running 4-6S. Your primary focus for parts is going to be weight. Racing frames are typically not meant for holding up to 100s of crashes. They sacrifice durability for weight. The bigger the weight, the more battery you'll need to complete your laps, which results in more weight. So Keeping weight down will ensure that you can be as quick as possible and complete your laps with as small a battery as possible. It's also worth finding a frame in which you can swap out the arms easily when you break them during a race. You want to have that drone back up and running by the next heat.

Long Range

Build a 5-7" Quadcopter. Your primary focus will be finding a frame that can carry an HD cam, and keeping weight down while being able to carry a large battery payload.

Cinematic Slow Smooth Video

Cinewhoop. Google It.

Simulators

You should be putting time in on a simulator before flying a drone in real life. It's going to save you a considerable amount of frustration and money. Your first time taking off is going to result in a crash, 100%. You may as well get those first 100 crashes out of the way on a simulator where it costs you zero time and money. You don't want to spend 10% of your time crashing and then 90% of your time on the workbench trying to band-aid it all back together.

You can use your Transmitter/Controller & even your goggles to gain real skill with learning to maneuver a drone around.

The 3 simulators I use, in the order in which I like them, are:

• Velocidrone
• DRL Simulator
• Liftoff

Join a Community

If you're trying to get started it's a really good idea to join Facebook Groups associated with whatever you're getting into. There are very active groups for almost anything out there. You can likely find a group for the parts you use to get help with them.

Here are some of the groups I'm in:

• BuckeyeFPV - A group from Columbus, Ohio
• Cincinnati Quad Racers
• Cinewhoopers - Small Ducted Drones for Cinemetography
• PyroflipRC - A company I buy parts from
• Betaflight Group

TODO:

• Add in something about Chargers, power sources, and accessories.

Complete Drones

• Spedix Rex 180 HD - $55
• DJI Mavic Pro Flymore Combo

Partial Drones

• Flynoceros Zeus HD Buid + Extras
• Hyperlite EVO HD Racing Drone
• SBA Spartan & Race-flight Drone

Flight Controllers

• New Airbot Nox 2.17 - $50

ESC

• Airbot Typhoon 32 4in1 - $15 - 30x30

Motors

• NEW Sunnysky R2305 2480kv - $40
• TMotor F60 Pro 2500kv (Set of 4) - $20

Frames

• RC Crazed Extra Slim Phast 5" (20mm Stack Only) - $18
• Sigan Drones x190 drone frame, PDB, and LEDs
• MRM Reaper & Shendrones Zoom Pod, with flaco wing suit and lots of extras

Accessories

• Brand new set of stock Taranis X9D gimbals - $15

Sold

• SOLD Think Tank FPV Radio Transmitter Cover - $15
• New FPV Camera - $5
• SOLD New Runcam Swift 2.8mm - $20
• Eachine VTX03
• RC Crazed Slim Phast 2.5" - $20
• SOLD Hyperlite 1106-7122kv Race Series Motors (Set of 4) - $49
• SOLD DYS 2205 2550kv with AIKON 30a ESC (set of 8) - $50
• SOLD Sunnysky 2305 2300kv Motors - heavily used - $15
• SOLD Edge Racing 2205 2480kv Motors - $10
• ~~SOLD Edge Racing 2204 2300kv Motors - $1
• 2x ORI32 4-in-1 ESC 20x20 - $25 - 20x20
• EMAX D-SHOT Bullet 12 A (Set of 4) - $30
• SOLD AIKON SEMF 30A ESC (Set of 5) - $15
• SOLD New Airbot Asgard 32 all-in-one - $45
• SOLD Omnibus F4 Pro Corner Flight Controller - $30

Let me start by showing you my current datamodel.prisma. I wanted to prune it down to reveal only what's relevant to the question but I worry in doing so, I take away context that could be important. So I apologize for it's length but here it is:

The Problem I ran into with just adding a new type Motor{} is when it comes to linking to the type MerchantLinks. A MerchantLink is essentially a way to attach a FlightController to a Merchant(store) with fields like price, link, availability. And as part of my DB Model I have it setup with relationships in both ways, with the purpose of being able to go to a Merchant page and see all of the products they offer. This means having a field called flightController on type FlightControllerMerchantLink. I cannot rename these to a product field on type ProductMerchantLink because product cannot be setup as product: FlightController || Motor || Battery || Camera || .... I can only point it at a single type.

Ok, so I'm not DB Design Engineer, and it's probably one of my weaker engineering skills so this is where I turn to colleagues and the internet. In doing so, as far as I can tell I have two options to get around this.

Option 1, Duplicate the FlightControllerMerchantLink Type for all product types

I think I could create a bunch of the type {ProductType}MerchantLink, one for each product type. And then link all of those as optional fields on the merchant, like so:

The thing I dislike about this approach is that it's not normalized. I'm repeating columns across many tables, such price, url, inStock, postedBy, etc repeated across all of the {ProductType}MerchantLink. Also, I'm not sure if this will have any ramifications that will block me from doing what I want to achieve in terms of site products, pages, or features. I've not taken this path yes, as I opted for option 2 instead. So I'll describe that now.

Option 2, Replace Flight Controller with Product and have a bunch of optional fields to store all the product-type-specific data

After talking with my co-worker he mentioned that he's solved this type of problem before by having a single type Product with a field of Type which describes what type it is, and then bunch of fields which point to the specific ProductType, where only one would be set at a time. Granted he did not implement something like this in Prisma, but I think it could still work. Like so:

With this setup, all of the linked Types for things common to all products, all link off of the type Product. These things include: Images, Product Reviews, Ratings, and Merchant Links. All of the fields common to all products can be added directly like cheapestPrice, averageReview, createdAt, etc.

I set about testing this new schema with a single type, FlightController and it seemed to work. So I pushed forward and expanded to 10 different product types.

Here's how my data model looks now

And here's my mutation resolvers for adding/editing products. NOTE: The code definitely needs cleaned up but I got it to a point that it works!

One of the benefits of going this route of making everything a base Type Product is that I can add/edit/delete any product type with a single mutation endpoint. I was also able to update my FE code for all of the pages I had setup for just flight-controllers to be retrofitted to work with the other product types.

This includes:

• Product List Page
• Product Details Page
• Product Edit Page
• Product Search Component
• Add/Edit Images Component
• Add/Edit Reviews Component

I'm not sure if this approach will cause issues down the road but it seems to be working for now.

One thing that I thought for sure wouldn't work but was pleasantly surprised on, was that my product list page queries still work, with a little modification. I'm able to sort/filter at the Product and Product.SubType (Product.flightController) levels in a single Prisma query like so. A key thing to note here is that I'm doing an orderBy at the Product level but I'm doing filtering where clause at the Product.flightController level.

Fast forward 2 weeks and this approach is still going strong. I created a new feature which allows the user to build an entire drone attaching all of the common components such as Frame, Motors, Flight Controller. The components are all of type Product, with product-specific sub-types. I plan to continue heading down this path and I'll report back if I run into any issues.

Key Terms:

AWS - Amazon Web Services

AWS S3 - Amazon Simple Storage Service is a service offered by Amazon Web Services that provides object storage through a web service interface. This is where our static SPA is hosted.

AWS API Gateway - A service for creating, publishing, maintaining, monitoring, and securing REST and WebSocket APIs. It gives us a public domain for attaching the SSL certificate.

AWS Certificate Manager - a service that lets you easily provision, manage, and deploy public and private Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with AWS services and your internal connected resources. SSL/TLS certificates are used to secure network communications

AWS CloudFront - a content delivery network offered by Amazon Web Services. It's how we create, and attach, the SSL certificate to our S3 bucket.

SPA - Single Page Application. In this case it will be a static Webpack-built ReactJS App.

Getting my EC2 instance, Node API, behind a domain

There are a bunch of different AWS Server/Service configurations to achieve this goal. They all come with pros and cons in terms of complexity, speed, scalability, flexibility, and cost. This project is a for-fun side project right now, so I don't want to set it up in a way that it could scale to a million users if it will cost me a fortune every month. Especially since it may very likely generate $0 indefinitely.

There are two ways I considered right off the bat. These are by no means the only two ways. If you know of a better, more cost-effective way, let me know in the comments.

1. Use API gateway to route the API traffic to the EC2 API and the UI traffic to the S3 bucket. API Gateway will be able to use the cert I generated in the last article. This is probably the preferred way but it comes at a +$16 minimum monthly cost just to turn on the API Gateway.
2. Drop the API Gateway and the Cloudfront Distribution and run all of the traffic to Nginx to an EC2 instance where my NodeJS is running. Then have Nginx be the SSL terminator and route the /api traffic to the API running on localhost:4000 and all other traffic to my S3 bucket for the front-end.

In either option, the browser would hit my site at mydomain.com and then it would make the API calls to mydomain.com/api.

I ended up going with Option 2, if only temporarily until this project is generating revenue. This meant throwing away some of the work I covered in the last article where I was using AWS CloudFront and AWS Certificate Manager to route traffic to my S3 bucket. Instead, the flow of loading the app will look like this.

This should have been pretty straightforward as there were 2 things that needed done:

1. Update the DNS of my website to include:

   A       @      IP_OF_EC2
   CNAME   www    @

2. Update Nginx to do the proper proxying

Embarrassingly, That took way longer than it should have because I kept ending up with dirty EOL characters all over and so I had to keep piping the changes through this tool every time. Which would collapse the file down with no spaces which made re-editing it a PITA. It was driving me nuts. Even when I'd edit it in Vim directly to try and remove the characters it still found them nested into my code in a way I couldn't seem to remove directly. I ended up fixing this by copying to "fixed' file back into Pycharm and installing this plugin and then highlighting the text and choosing "Reformat Code" which did the trick. I should have done this right off the bat instead of fighting through it, but I digress.

Once I had Nginx configured correctly, and the DNS changes had propagated to point my domain to my EC2 Public IP, the site was loading again on HTTP properly. The next step is adding SSL to the mix.

Get HTTPS Setup with Certbot

Since I"m on AWS Amazon Linux 2 kernel which uses YUM instead of APT, I started with this tutorial and followed up through step 5 where they start getting into Apache Configuration.

Next jump to step 4 here: https://certbot.eff.org/lets-encrypt/centosrhel7-nginx.html. At that point, you've got the certbot installed and you can proceed for a generic Nginx setup.

After wrapping that up, I was able to reload my site and it immediately got kicked over to HTTPS and all was well. I was kind of surprised to see even my WebSocket connection was using WSS:// without any issues.

🎉 WOOHOO! 🎉

Key Terms

AWS - Amazon Web Services

AWS S3 - Amazon Simple Storage Service is a service offered by Amazon Web Services that provides object storage through a web service interface. This is where our static SPA is hosted.

AWS Certificate Manager - a service that lets you easily provision, manage, and deploy public and private Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with AWS services and your internal connected resources. SSL/TLS certificates are used to secure network communications

AWS CloudFront - a content delivery network offered by Amazon Web Services. It's how we create, and attach, the SSL certificate to our S3 bucket.

SPA - Single Page Application. In this case it will be a static Webpack-built ReactJS App.

This week I started on the topic of moving my project from http to https to make it more legit as I near a public "release".  Why is moving to HTTPS important?

Reason 1: Google

The big driver for this need is Google, which is phasing support for HTTP connections out of Google Chrome. Today, Chrome warns users when they’re accessing a site HTTP, but doesn’t prevent it altogether. In some cases, they get a nasty warning which will likely scare away my users.

Reason 2: It’s the Right Thing to Do

HTTP connections reveal, at a minimum, full information about the routing and content of requests to everyone else on the local network. If your users are accessing your site in a coffee shop, no amount of server-side security will prevent you from leaking information in the stage between the user (and their router) and your site. Now I won't have any sensitive data, but it's a good practice to follow regardless.

So let's get started!

Step 1: Get a Cert

I initially all signs pointed toward https://letsencrypt.org/getting-started/ and the tool https://certbot.eff.org/. These tools looked great but as far as I could tell they aren't a good fit if you're deploying a static site running on Amazon S3 because you don't have SSH access as far as I know. Besides, AWS has its own tooling for setting up SSL certs.

Enter AWS Certificate Manager

Amazon Web Services (AWS) has great resources for issuing and using SSL certificates, but the process of migrating existing resources to HTTPS can be complex — and it can also require many intermediate steps. It ended up being more work than I had expected, but mostly because GoDaddy DNS and UIs are awful.

To get started you log into the AWS Certificates Manager console and request a certificate. They have 2 options for confirming your domain:
* CNAME Records
* Email

The documentation suggests the CNAME path is the better of the two long term, so that's what I did. They give you some CNAME records you must add to your DNS Management for your domain. In my case this is GoDaddy (But not for long hopefully).

I entered the CNAME records, set the TTL to 30 min, and then waited.... and waited.. and waited.

After 24 hours I started to get impatient. I started googling around and found this page which details an issue with the GoDaddy DNS system. The problem with GoDaddy is that they auto-append the domain name at the end of the "host" field when creating a CNAME record. Therefore, you are required to omit the apex domain from the "name" string provided to you by ACM when adding it to "host" on GoDaddy.

So if AWS gives you a record like:

RECORD NAME: _ho9hv39800vb302vejvnw3vnewoib3u.example.com.
RECORD VALUE: _cjhwou20vhu20uvoneouw20vuyb2ovb9.j9s73ucn9vy.acm-validations.aws.

Enter the following for GoDaddy:

NAME: _ho9hv39800vb302vejvnw3vnewoib3u
VALUE: _cjhwou20vhu20uvoneouw20vuyb2ovb9.j9s73ucn9vy.acm-validations.aws.

After doing so, I popped back over to AWS Cert Manager and hit refresh and after a couple of minutes, it went from pending to verified!

Step 2: Link with CloudFront

To connect your new cert with your S3 bucket you need to add a product to sit between your DNS & your S3 bucket. This product is AWS CloudFront

Go through the AWS Console and access the CloudFront product. Click to create a CloudFront distribution and select your AWS S3 Bucket as an origin and choose the cert we just created above.

After doing this, I tried to access the site via the CloudFront distribution Domain Name and it told me Access Denied.

It looks like I need to specify the Default Root Object as index.html which is the root file inside my S3 bucket:

When I refreshed my page, it loaded the HTML document stored in the S3 bucket, but the page was blank. I checked the JS console and saw it immediately threw errors. client.js:653 Mixed Content: The page at 'https://d1krmsds175gpf.cloudfront.net/' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint 'ws://54.xxx.xx.49/'. This request has been blocked; this endpoint must be available over WSS.

I believe this is because my S3 bucket is now being served over HTTPS but the URI of my EC2 instance/api I'm trying to connect to is hardcoded to HTTP and :WS. I updated my FE to use WSS but it's telling me that my endpoint doesn't have WSS enabled still.

Watch Out! I learned that CloudFront has a built-in cache, so changes I was making to the S3 bucket weren't showing up when I refreshed the CloudFront URL. I ended up finding this post which describes going in and setting the Min/Max TTL. I'll need to make sure I set this back later to take advantage of caching.

To at least verify that this would work on HTTP, I bumped the CloudFront Distribution viewer Protocol Policy to: HTTP and HTTPS and loaded the page on HTTP and verified it worked.

Now I need to figure out how to get my AWS EC2 instance to use an SSL Cert. A cert typically needs to be applied to a domain, not an IP, which is how I'm accessing my API. So this needs to change I think.

Up next I made some changes in order to get my NodeJS GraphQL API, running on AWS EC2, setup with an HTTPS Certificate.

• Your standard ghost.org site which really doesn't need a domain. I spun up a standard DigitalOcean Ghost.org droplet and I access that blog site through the IP address. This won't be a customer facing site but rather a content authoring tool.
• Your Gatsby Site which pulls in content from the blog, but is the full Gastby platform. I plan to deploy this one via Netlify and point a domain at it. It'll be the customer facing site.

UPDATE: 10/2019 - This setup ended up being pretty easy and I was able to do exactly as described above. I meant to document this process but unfortunately forgot about it, and it's been over 8 months since I got the work done. Feel free to ask me question though and I'll help out where I can!

Here's the deployed project: https://www.moonlitaerial.com/

What is Lerna?

From the docs: "A tool for managing JavaScript projects with multiple packages." When I first read this it didn't make complete sense so I had to dig in further.

Lerna is an NPM installable tool that you install on top of your existing project. When you init it, it adds some extra directories/files and structure to your app which enables some handy tools which you can run against your projects to add some quality-of-life features.

What kind of QoL features?

• It can do some fancier npm-link functionality for allowing you to reference other packages in your mono-repo via symlinks. This is helpful since in a monorepo you cannot npm install from github links npm i http://github.
• It can help you run scripts across all of your packages. For instance learna run test will kick off the npm run test in all of your packages. This can be used for other scripts like for building your code.
• It can help you with creating new versions of your packages for releasing code with some intelligent features like only updating versions when necessary.
• It will maintain a Changelog keeping track of everything that is in a release.
• It will give you a diff tool which enables you to see the changes in all commits across all packages since the last release.

Who should use Lerna?

If you run a Monorepo it might be worth using Lerna. If you don't use a monorepo I doubt it's worth adding. In my current role I don't believe it's a good fit for what we do, as we have the opposite of a Monorepo. Our frontend stack is a MicroServices architecture.

If I've missed anything please drop me a message in the comments below. Thanks!

Here's what I got on the initial run of npm audit:
`found 239 vulnerabilities (1 moderate, 238 high) in 42756 scanned packages`

....................
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ lodash                                                       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ @private/ui-scaffolding                                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ @private/ui-scaffolding > karma-webpack > lodash                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://npmjs.com/advisories/1065                            │
└───────────────┴──────────────────────────────────────────────────────────────┘


┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ lodash                                                       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ @private/ui-scaffolding                                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ @private/ui-scaffolding > react2angular > ngcomponent > lodash  │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://npmjs.com/advisories/1065                            │
└───────────────┴──────────────────────────────────────────────────────────────┘


┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ lodash                                                       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ @private/ui-scaffolding                                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ @private/ui-scaffolding > webpack-dev-server >                  │
│               │ http-proxy-middleware > lodash                               │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://npmjs.com/advisories/1065                            │
└───────────────┴──────────────────────────────────────────────────────────────┘


found 239 vulnerabilities (1 moderate, 238 high) in 42756 scanned packages

Ideally, you could use npm audit --fix to have it auto-fix the issues. Unfortunately, there's a good chance this isn't going to work for you as I don't believe it will update past any MAJOR versions.

The reason this ended up being a lot of work was that it usually wasn't the immediate dependency that had the vulnerability. It was more likely a dependency 2-6 levels deep that was the problem.

Here's an example dependency tree describing that problem.

Your Project Uses> Third-Party-Lib-A@1.0.0 Uses> Third-Party-Lib-B@1.0.0 Uses> Third-Party-Lib-C@1.0.0 Uses> Third-Party-Lib-D@1.0.0

Let's say Project D had a security vulnerability that got patched out in version 2.0.5 but  Third-Party-Lib-C@1.0.0 was pinning it to version 1.0.0. Well,  Third-Party-Lib-C is going to have to update itself to use version Third-Party-Lib-D@2.0.5. But maybe it's on version 4 now. This means that for Third-Party-Lib-B to get rid of it's security vulnerabilities it's going to require a potential refactor to pull in Third-Party-Lib-C@4.0.1. Now just continue that pattern up the chain until you get to your code. So you might have been on Third-Party-Lib-A@1.0.0 but now to get rid of the security vulnerability, you'll have to upgrade to whatever version  Third-Party-Lib-A ended up at to patch out its downstream issues. This may come with a ton of breaking changes as which will require refactoring of your own code's implementation.

How long will it take for all of these 3rd party libraries to get their dependencies updated? LibA can't do anything until B fixes its dependencies, B is blocked by C, and C by D. This seems like a real cluster-f and if I'm not understanding it right, please let me know in the comments.

It gets scarier. What if any one of those dependencies along the way is no longer maintained by anyone? You have to find an alternate library to use that is maintained, or fork a library just to make these security updates, and then convince the upstream dependencies to use your library instead. And now you're its maintainer. 😳

If the project is maintained it may mean that I have to open PRs with those open source projects and fix the issues, which I truly don't have the cycles to do.

Luckily for me, In most cases, I was able to simply upgrade the immediate dependencies, IE Third-Party-Lib-A, to the latest libraries and that resolved most of the downstream issues. I also ended up removing a lot of unused dependencies and replaced dependencies that were depreciated. Security vulnerabilities aside, this was a good exercise to keep our codebase lean.

This would have gone more smoothly except that I manage a micro-services-frontend architecture which meant that I had 30'ish frontends with dependencies to fix. Though most of these libraries share a lot of the same dependencies. Also, we run these frontends in Docker containers and I had a misconception on what layers are cached. So even after fixing some of the dependency issues, and re-running npm-audit, I was getting a lot of false negatives as my changes weren't picked up.

It was a good bit of work but we now have all of our front-end repos vulnerability free! In order to enforce this going forward, we've done two different things:

1. We added some lines to our build tools which block PRs from being merged. Essentially on each push to the branch, it runs some checks like lint, unit-tests, and now npm-audit . If it detects any issues it will fail the build and notify the developer that action must be taken.
2. We pin all of our 3rd-party dependencies. This is to prevent random build failures on a Developer's PR when he/she didn't even modify dependencies. Also it gives us the security that we'll never update a dependency to the latest version before the community and NPM team have a chance to catch any potential issues with it. Also, there's no sense in updating working code in production just for the sake of updating. We may as well only update the library when there's a feature or bugfix that was made that we require.

In ProjectA I get found 0 vulnerabilities.

In ProjectB  I get the following:

....................
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ lodash                                                       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ @private/ui-scaffolding                                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ @private/ui-scaffolding > karma-webpack > lodash                │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://npmjs.com/advisories/1065                            │
└───────────────┴──────────────────────────────────────────────────────────────┘


┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ lodash                                                       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ @private/ui-scaffolding                                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ @private/ui-scaffolding > react2angular > ngcomponent > lodash  │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://npmjs.com/advisories/1065                            │
└───────────────┴──────────────────────────────────────────────────────────────┘


┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ lodash                                                       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ @private/ui-scaffolding                                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ @private/ui-scaffolding > webpack-dev-server >                  │
│               │ http-proxy-middleware > lodash                               │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://npmjs.com/advisories/1065                            │
└───────────────┴──────────────────────────────────────────────────────────────┘


found 239 vulnerabilities (1 moderate, 238 high) in 42756 scanned packages

All of the 239 vulnerabilities are found as dependencies of ui-scaffolding and all of the vulnerabilities are related to lodash not being updated to version 4.17.12 or later. So why do I get all these in 1 repo and not they other when they use the same library?

Wait I figured out why

Ok.. after all this, I realized that the problem of 1 project works, one doesn't that really all I needed to do was delete the package-lock.json file in the failing web repo and rerun npm i would actually result in found 0 vulnerabilities. How does this work.. No idea. Previously changing node modules in the package.json file and running npm i always generated a new lock file. So I'm clueless as to why this wasn't working as expected. I hope to revisit this Monday when I'm back in the office and try and get to the bottom of what was going on, and I'll update this page accordingly. Because right now it'll just confuse people.

Update a few weeks later: Ok I know what was going on.

• If you have node modules installed that satisfy the lock file, then it's not going to go grab the latest ui-scaffolding version.
• If you run npm i and it detects changes to the package.json file, it'll generate a new lock file to the latest satisfying version.

I also need to look into https://www.npmjs.com/package/npm-audit-ci-wrapper