DAILY NEWS

Stay Ahead, Stay Informed – Every Day

Advertisement
System Design Tradeoffs – DEV Community


System Design Tradeoffs

Scaling

Vertical vs Horizontal Scaling
Scalability vs Performance

Consistency & Availability

Consistency vs Availability (CAP)
Strong vs Eventual Consistency
ACID vs BASE

Data & Storage

SQL vs NoSQL
Normalization vs Denormalization
Read-Through vs Write-Through Cache

Communication & Processing

Synchronous vs Asynchronous Communication
Batch vs Stream Processing
Long Polling vs WebSockets vs SSE
Push vs Pull Architecture
REST vs GraphQL vs gRPC
REST vs RPC

Architecture

Monolith vs Microservices
Stateful vs Stateless Design
Concurrency vs Parallelism

Performance

Latency vs Throughput
Performance vs Cost

🔗 Connect with me on LinkedIn:Let’s dive deeper into the world of software engineering together! I regularly share insights on JavaScript, TypeScript, Node.js, React, Next.js, data structures, algorithms, web development, and much more. Whether you’re looking to enhance your skills or collaborate on exciting topics, I’d love to connect and grow with you.

Follow me: Nozibul Islam



Source link

Stop Writing Endpoints. Start Defining Systems.



For a long time, I thought building APIs meant writing endpoints.

You know the pattern:

Define a route
Validate input
Query the database
Transform the result
Send a response

Do that over and over again.

Different routes. Same structure.

The Illusion of Control

Writing endpoints feels productive.

You’re in control of everything:

The logic
The validation
The data flow

But after a while, something becomes obvious:

You’re not building systems.

You’re repeating patterns.

The Real Problem

Most APIs look like this:

app.get(‘/users/:id’, async (req, res) => {
const id = req.params.id;

if (!id) {
return res.status(400).json({ error: ‘Missing id’ });
}

const user = await db.users.findById(id);

if (!user) {
return res.status(404).json({ error: ‘Not found’ });
}

return res.json(user);
});

Enter fullscreen mode

Exit fullscreen mode

Now multiply that by:

Dozens of endpoints
Multiple resources
Different validation rules
Slight variations in logic

You end up with:

Repeated code
Inconsistent patterns
Hard-to-maintain systems

You’re Not Writing Logic. You’re Rewriting Structure.

Look closer at most endpoints.

They follow the same shape:

Extract input
Validate input
Execute query
Handle errors
Return response

The structure doesn’t change.

Only the details do.

So why are we rewriting the structure every time?

The Shift: Define, Don’t Rewrite

Instead of writing endpoints…

Define them.

What if your API looked like this instead?

get:
user:
GetUserById:
input:
id: number
where:
id: $param.id
response:
id: number
name: string
email: string

Enter fullscreen mode

Exit fullscreen mode

No route handler.

No repeated boilerplate.

Just a definition.

What This Changes

When you define systems instead of writing endpoints:

Structure becomes consistent
Validation becomes automatic
Queries become predictable
Behavior becomes visible

You’re no longer guessing how something works.

You can read it directly.

From Endpoints to Systems

Traditional approach:

Every endpoint is custom
Logic is scattered
Behavior is implicit

System-driven approach:

Endpoints follow a pattern
Logic is structured
Behavior is explicit

You move from “code-first” to “contract-first.”

Where the Code Goes

This doesn’t eliminate code.

It moves it.

Instead of writing endpoint logic repeatedly…

You write:

A compiler that reads definitions
A pipeline that executes them
A system that enforces rules

Code becomes the engine.

Not the repetition.

Example Flow

With a system-driven approach, a request might flow like this:

Request → Parse Definition → Validate → Build Query → Execute → Format Response

Enter fullscreen mode

Exit fullscreen mode

The difference is:

The flow is constant
The behavior is defined in configuration

Why This Matters

Without this approach:

Every developer writes endpoints differently
Bugs are repeated across routes
Refactoring becomes painful

With this approach:

Patterns are enforced
Behavior is predictable
Systems scale cleanly

“Isn’t This Less Flexible?”

Yes.

And that’s the point.

Unlimited flexibility leads to:

Inconsistency
Complexity
Fragile systems

Constraints lead to:

Where This Fits

This kind of system works best when:

You have repeated CRUD patterns
You want consistent APIs
You care about long-term maintainability

It doesn’t replace every use case.

But it replaces most of the boring, repetitive ones.

The Bigger Idea

This isn’t just about APIs.

It’s about how we build software.

Instead of:

Writing everything manually
Repeating patterns
Hoping for consistency

We can:

Define systems
Enforce structure
Let the engine handle execution

Final Thought

Writing endpoints feels like control.

But it’s often just repetition.

Defining systems feels restrictive at first.

But it leads to something better:

Clarity.

Consistency.

Scalability.

That’s why I stopped writing endpoints…

…and started defining systems.



Source link

29 Zapier + Make automations replaced in four weeks


The bill that I no longer understood One morning in March, I reread the list of recurring direct debits from L’Atelier Palissy, and I came across a line that surprised me by its regularity: Zapier Pro, $73.50, every month for eighteen months. Next to it, a smaller but equally discreet Make Pro. I count: twenty-one active Zaps, nine Make scenarios, a few stopped, a few obviously broken for weeks. Nobody noticed, because each automation lived on its own dashboard, with its own history, its own logs that no one ever looked at. I didn’t know exactly what each one did. I knew there were about thirty of them, that they passed data from Formidable to Mailchimp, from Meta Lead Ads to a shared Google Sheet, from Stripe to a confirmation email, that they triggered on webhook, on polling, on schedule, and that half of the silent system failures probably came from there. I didn’t have the energy to go look, because going to look meant opening a tool in which I wasn’t at home. Twenty-eight days later, it was gone. Zero Zap, zero Make scenario, everything replaced by three hundred lines of TypeScript running in my Rembrandt ERP, under Sentry, under tests, with a single overview that I look at in the morning at coffee. If you have 30 seconds. Zapier/Make automations pay three invisible debts: they live outside your database, they log elsewhere than your monitoring, and they are triggered by rules that we end up no longer reading. Replacing thirty automations with a single event pipeline takes four weeks, not six months, provided you follow a golden rule: never cut a Zap before having validated its replacement in double writing for three to five days. The article gives the method, timetable and cost avoided. Three invisible debts The first debt that no-code tools produce is called, in internal literature which does not yet exist, distributed debt. Your data lives in three places at once, and none of the three are canonical. My CRM thought the truth was in Google Sheets. Mailchimp believed the truth came from Zapier. Supabase didn’t think anything because I only wrote part of what was happening there. The day a lead lands in three different tabs with three different spellings, no one knows which record is the real one. The only way to decide is to choose a place that wins by construction, and do everything else as a slave. The second debt is the absence of unified monitoring. A broken Zap sends an email to the address that created the Zapier account, possibly to no one. A Make scenario that fails in its third step leaves the first two consuming quotas, and the only trace is a small red number in a dashboard that no one opens. Sentry, Datadog, Grafana — none aggregate. You learn that your automation is dead when a customer calls to say that they have not received their confirmation email. The third debt is the most silent, and it is that of the rules that we forget to have written. A Zap created eleven months ago to manage a particular case of the summer season continues to run the following winter, and it routes a Paris lead to the email address of a colleague who has left the house. You don’t see it, because the lead is still coming, apparently. Nobody rereads a Zap. It’s done so that we don’t have to read it. This is precisely what makes it a debt. The golden rule that took me two weeks to accept. My first attempt, at the beginning of April, was naive. I’d write a Rembrandt replacement, cut the Zap, move on to the next one. My third attempt ended in Slack at 11 p.m., a Meta lead lost because my webhook was not deployed on the correct Vercel environment, and the certainty that if I continued like this, I was going to break at least one critical element before the end of the week. I set the following rule, which I kept until the last Zap. Never cut a Zap before having validated its replacement in double writing for three to five days. Concretely: I write the replacement, I deploy it, it runs at the same time as the Zap. Each lead arrives in duplicate in my Supabase, and in the email address of the team that receives the notification. For three to five days, I compare: same lead, same data, same timings, same emails sent? If so, I cut the Zap. If not, I disable my code and I still have a spinning net while I figure out what broke. The worst possible side effect during the transition is a lead received twice by the sales team. It’s an annoyance, not a disaster. A lost lead is a silent catastrophe that we discover a month later. The architecture that replaced the thirty automations The target is of a simplicity that was not visible as long as I stayed in the no-code tools. Meta Ads ──┐ Formidable ──┤ Stripe ──┤──► Webhooks Rembrandt ──► Supabase (single source) Manual ──┘ │ ├── Gmail SMTP (internal notifications) ├── Slack (team alerts) ├── Meta CAPI (campaign feedback) ├── automation_logs (traceability) └── Cron → Mailchimp then Brevo Enter fullscreen mode Exit fullscreen mode A single entry point per source, a single storage location, a parallel fan-out to the notification tools. The core is contained in a file called lib/lead-pipeline.ts which executes the outgoing integrations in parallel after each insert in the contacts table. export async function runLeadPipeline(lead: Lead) { await Promise.allSettled(( syncMailchimp(lead), notifySlack(lead), notifyGmail(lead), sendMetaCapi(lead), generateFirstContactDraft(lead), )) await logAutomation(lead, /* status by tool */) } Enter fullscreen mode Exit fullscreen mode Promise.allSettled rather than Promise.all because if Slack is down, I still want to send the email. Each result feeds an automation_logs table which is the only thing I look at in the morning: how many leads arrived, which tools succeeded, which failed, over what time window. The schedule as it really happened Week What I did Zaps cut S1 Central pipeline + Slack client + automation_logs table 0 S2 Great direct in double write 0 S3 Cut of ten Zaps Great + webhook Meta in duplicate 10 S4 Cut of eight Zaps Meta + webhook Stripe 18 S5 Scenarios Make (PDFs, crons) and cleanup 21 S6 Kill Zapier Pro, planned downgrade 21/21 The day of the final cut, I didn’t sleep the night before, and the next day I opened my automation_logs dashboard every hour. Thirteen days later, nothing had broken. Today I still maintain a dead route, sync-gsheets-leads, which I never call but which serves as a reactivable net until the end of the month. What we gain that we never suspected The economic gain is obvious – around a hundred euros per month in consolidated subscriptions. But what surprised me was a gain in understanding. The first week after the cut, I found that I understood my system for the first time in eighteen months. I could open a file, reread a routing rule, modify it, test it, deploy it in twenty minutes. Before, even a trivial modification – changing the recipient address of a notification email – went through five Zapier tabs and a dull fear of breaking one by moving another. Two short scenes come to mind. The first, I had to call Gaspard, our IT service provider, to retrieve the password for the Zapier account. He had it, he gave it to me, he didn’t ask why I wanted to go. The second, earlier in the morning, Françoise came out of her office with her cup in her hand and stood in front of mine: “Good. How long do you plan to keep your Meta duplicates? Because Hélène receives two emails for the same lead, she is starting to get annoyed. » It was the third week, I told her “Two more days”, she agreed, put down her cup, and the cup was made the next evening. I learned that double writing has a cost in team patience that should neither be minimized nor stretched beyond what is necessary. There is a particular hygiene to having a system held in one place. We underestimate it until it is there, because no-code tools sell precisely the promise that it is everywhere, that it no longer has to be thought about. The truth is that if it’s not thought of in one place, it’s just buried. It costs less to write, and much more expensive to live. What you can copy into your project Reusable patterns extracted from this migration, independent of my stack: The golden rule — double writing three to five days before cutting. Not negotiable. A duplicate lead is better than a lost lead. The additional time is paid once and is reimbursed for each incident avoided A unique event pipeline — a runPipeline(event) function called after each insert, which executes the outgoing integrations in parallel (Promise.allSettled) and traces the result of each in a dedicated table An automation_logs table — one row per event, one column per outgoing tool with its status. It’s the only dashboard we look at in the morning, and it replaces all the separate dashboards for no-code tools. A reactivable post-cut net — keeps the road dead for two more weeks. The day a bug surprises you, it takes fifteen seconds of vercel.json to put the net back together. Afterwards, you delete for good And a broader discipline: any tool that houses your business logic outside your database makes you pay three debts — distributed, without monitoring, unreadable. Zapier and Make are useful for prototyping. They become dangerous as soon as serious activity depends on them. How many no-code automations are running on your system right now, and when was the last time someone reread them all? I read the comments. Companion code: rembrandt-samples/lead-pipeline/ — runLeadPipeline with Promise.allSettled, automation_logs schema, hub-and-spoke diagram, MIT, ready to copy.



Source link