20 Dollars a Month vs 20 Commits

9/2024 | Aman Azad

18-ish hours, 28 commits, and 600+ lines of terraform code later I have a fully automated CI/CD pipeline for a side project built out in AWS using all the latest and greatest offerings - CodePipeline with a CodeStar source connection, security groups to allow just the right traffic through to my ECS containers, etc.

All this to say I could've just deployed to Vercel for $20/mo.

Fresh off going through the journey to self host NextJS on AWS, I wanted to run through what I think the trade-offs are.

The Shadow

Building my startup I learned that you get 80 to 95% of the CI/CD pipeline done during the first go around. The remaining % points is a lurking shadow that sits between a spectrum of things my pipeline actively needs updating, and a more sinister, lingering question if I set up my autoscale policies correctly, or if I allocated enough CPU/memory to my containers, or if -

So while it feels good to have the pipeline I built work similar to what a Vercel or a Fly.io could do, there's an unseen cost of maintaining it. Time and mental overhead that for sure could've been better spent on product.

And as far as costs are concerned I still have to pay AWS for my hosting so self hosting is not a total panacea for cost savings, though I do have far more control of my AWS resources.

Also taking a look at Vercel's pricing page and running it against what's provided in the Pro plan, there's nearly $409/mo worth of services offered for the $20/mo cost.

Cognitive Framework Dissonance

Middleware, cache, internationalization, and more? Maybe?

NextJS gets confused, or acts unexpectedly when selfhosting. I like to call this "cognitive framework dissonance", specifically around NextJS's edge features.

The framework operates on an assumption that it's being deployed in an optimal environment (I think? IMO).

Middleware

You know and I know that I'm running this thing on a Docker container. NextJS will not stop reminding you about doing node things in it's middleware, and you have to make silly workarounds to get it to stop erroring out. Currently I have my middleware.ts file essentially act as a reverse proxy to a another Node endpoint to run auth, and a few other things I need to do.

Not a huge miss - but certainly not ideal.

As NextJS continues to develop I'm sure there'll be more and more instances of running the framework outside the expected environment to cause issues.

Cache

The caching features is a great example here - though the in-memory caches NextJS operates for fetch calls works great in Docker, there's a hidden nuance in here. The in memory cache for the built-in override to the fetch API will not be shared between containers. You'll have to configure a custom cache handler like this to get it working properly.

Internationalization

There are some built in defaults with internationalization that are also a root cause for seemingly random behavior. This cropped up when I was chatting with an e-commerce site trying to troubleshoot seemginly random redirects and trailing slash inconsistencies - big big problem for SEO.

Turns out on the pages router there's an internationalization helper that needed to be explicitly turned off in order to not have redirects. Took some digging but the setting was here.

So that's what I found so far - will there be more framework caltrops? Who's to say - though I am confident in the NextJS team to keep chiseling away at the rough edges.