From 1.2s to 180ms: how we refactored microservices to handle 4x more requests

Cutting latency isn't just about making an API faster. In distributed architectures, improving response time can reduce CPU usage, lower concurrent connections, take pressure off the database, shrink internal queues, increase throughput per Pod, and ultimately cut infrastructure costs.

In this article, we'll walk through a reference scenario where a microservices-based API had an average latency of roughly 1.2 seconds and, after a series of refactorings, ended up responding in close to 180 ms, handling roughly four times more requests on similar infrastructure.

These numbers are used as a technical case study. They don't represent an automatic or universal outcome: the real gain depends on the architecture, language, database, traffic pattern and existing bottlenecks.

The core idea is to walk through a full evolution:

Problem → Profiling → Refactoring → Database → Cache → Concurrency → Kubernetes → Observability → Production

1. The problem: the API scaled infrastructure, but not performance

Imagine a SaaS platform running hundreds of thousands of operations per hour.

The main architecture uses:

• Kubernetes;
• Amazon EKS;
• Node.js APIs;
• PostgreSQL;
• Redis;
• RabbitMQ;
• Prometheus;
• Grafana;
• Loki;
• OpenTelemetry;
• KEDA and HPA for scalability.

An API called orders-api handles requests to query and create orders.

Under normal load:

500 req/s

During peaks:

1,500-2,000 req/s

The application initially showed:

p50: 650 ms
p95: 1,200 ms
p99: 2,800 ms

As traffic grew, the team's go-to fix was adding Pods.

6 pods
↓
10 pods
↓
15 pods
↓
20 pods

Even so, latency stayed high.

The problem was the assumption that:

more replicas always means more capacity.

That's not necessarily true.

If every Pod keeps competing for the same database, running inefficient queries and holding blocking operations, adding replicas can simply shift the bottleneck elsewhere.

2. What it means to refactor microservices for performance

Refactoring a microservice for performance means changing its implementation and, eventually, its internal architecture without necessarily changing its functional behavior.

The goal isn't just prettier code.

It can involve:

• reducing CPU time;
• reducing external calls;
• eliminating redundant queries;
• reducing the amount of data transferred;
• reducing network round trips;
• parallelizing independent operations;
• removing synchronous processing;
• improving connection management;
• using cache properly;
• reducing database contention;
• optimizing serialization;
• avoiding unnecessary allocations;
• increasing throughput.

One of the most important concepts in this process is distinguishing latency from throughput.

Latency

The time needed to process a single operation.

Example:

request → 180 ms → response

Throughput

The number of operations the system can process in a given interval.

Example:

2,000 requests per second

An API can have low latency and low throughput.

Likewise, it can process many operations per second while individual latencies stay high.

An efficient architecture usually aims to balance both.

3. Key concepts involved

3.1 p50, p95 and p99 latency

Using only the average can hide problems.

Imagine:

90 requests → 100 ms
10 requests → 4 seconds

The average might look acceptable, but 10% of users will have a bad experience.

That's why we use percentiles.

p50

50% of requests finished below that value.

p95

95% of requests finished below that value.

p99

99% of requests finished below that value.

In production, p95 and p99 are frequently more useful than the average.

4. Before refactoring: find the bottleneck

Performance shouldn't be optimized based on assumptions.

The first step should be instrumenting the application.

We can roughly break down a 1.2-second request:

API Gateway / Ingress        15 ms
Middleware                   20 ms
Auth                         45 ms
PostgreSQL                  620 ms
External service            280 ms
Processing                   75 ms
Serialization                 20 ms
Network                      25 ms
-----------------------------------
Approximate total         1,100 ms

This analysis completely changes the strategy.

Optimizing JSON serialization from 20 ms to 10 ms is unlikely to fix anything if the database is eating up more than 600 ms.

The first rule of optimization is:

optimize the component that's actually eating up time.

5. Initial architecture

The original flow could be represented as:

Client
   ↓
Load Balancer
   ↓
Ingress Controller
   ↓
orders-api
   ↓
users-api
   ↓
products-api
   ↓
PostgreSQL
   ↓
External API
   ↓
Response

The problem with this architecture is the number of synchronous dependencies.

To answer a single request, orders-api had to wait on several services.

Something close to:

Request
   ↓
orders-api
   ↓
users-api ────────── 120 ms
   ↓
products-api ─────── 180 ms
   ↓
PostgreSQL ───────── 500 ms
   ↓
External API ──────── 250 ms
   ↓
Response

Even small individual latencies add up.

This effect is frequently called latency amplification.

6. First refactor: eliminating sequential calls

Consider this code:

const user = await getUser(userId);
const products = await getProducts(productIds);
const inventory = await getInventory(productIds);

If each operation takes:

getUser       100 ms
getProducts   150 ms
getInventory  180 ms

The approximate total will be:

100 + 150 + 180 = 430 ms

But if these operations are independent, they can run at the same time.

const [user, products, inventory] = await Promise.all([
  getUser(userId),
  getProducts(productIds),
  getInventory(productIds),
]);

Now the time tends to be close to the slowest operation:

~180 ms

instead of:

~430 ms

This doesn't mean everything should be parallelized indiscriminately.

If dozens of queries fire simultaneously, we might just shift the problem to PostgreSQL, Redis or external APIs.

Concurrency needs to be controlled.

7. Second refactor: reducing database queries

One of the biggest problems found in microservices is the N+1 queries pattern.

Imagine:

const orders = await db.query(
  `
  SELECT *
  FROM orders
  WHERE customer_id = $1
`,
  [customerId],
);

for (const order of orders.rows) {
  order.items = await db.query(
    `
    SELECT *
    FROM order_items
    WHERE order_id = $1
  `,
    [order.id],
  );
}

If the user has 100 orders:

1 query for orders
+
100 queries for items
=
101 queries

In production this can be extremely expensive.

An alternative is a single query:

SELECT
    o.id,
    o.created_at,
    o.status,
    oi.product_id,
    oi.quantity,
    oi.price
FROM orders o
LEFT JOIN order_items oi
    ON oi.order_id = o.id
WHERE o.customer_id = $1
ORDER BY o.created_at DESC;

Another option is two controlled queries:

SELECT *
FROM orders
WHERE customer_id = $1;

and:

SELECT *
FROM order_items
WHERE order_id = ANY($1);

This reduces:

101 queries

down to:

2 queries

8. Indexes: a few milliseconds can turn into seconds

Consider:

SELECT id, status, created_at
FROM orders
WHERE company_id = $1
AND status = $2
ORDER BY created_at DESC
LIMIT 50;

Without an adequate index, PostgreSQL might run a:

Seq Scan

over millions of rows.

A possible index would be:

CREATE INDEX CONCURRENTLY idx_orders_company_status_created
ON orders (company_id, status, created_at DESC);

On large tables, CONCURRENTLY reduces the impact on write operations while the index is being built.

Before adding indexes, use:

EXPLAIN ANALYZE
SELECT id, status, created_at
FROM orders
WHERE company_id = 100
AND status = 'pending'
ORDER BY created_at DESC
LIMIT 50;

The goal is to understand the real execution plan.

Adding indexes indiscriminately also has a cost:

• it takes up storage;
• it increases INSERT work;
• it increases UPDATE work;
• it adds maintenance overhead;
• it increases database cache usage.

9. Third refactor: return only the data you need

Another common antipattern is:

SELECT *
FROM orders;

That query can pull back:

• audit fields;
• large JSON blobs;
• metadata;
• descriptions;
• data that won't even be used.

Prefer:

SELECT
    id,
    customer_id,
    status,
    total,
    created_at
FROM orders
WHERE company_id = $1
LIMIT 50;

Less data means:

less disk reading
→ less memory
→ less network traffic
→ less serialization
→ shorter response time

10. Fourth refactor: cache used correctly

Not every query needs to hit PostgreSQL.

Imagine an endpoint:

GET /products/123

Product data changes a handful of times a day but gets queried thousands of times per minute.

Redis can be used:

Request
   ↓
Redis
   ├── HIT → response
   │
   └── MISS
        ↓
    PostgreSQL
        ↓
    Redis SET
        ↓
    response

Node.js example:

async function getProduct(productId) {
  const key = `product:${productId}`;

  const cached = await redis.get(key);

  if (cached) {
    return JSON.parse(cached);
  }

  const result = await db.query(
    `
    SELECT id, name, price, status
    FROM products
    WHERE id = $1
    `,
    [productId],
  );

  if (!result.rows[0]) {
    return null;
  }

  await redis.set(key, JSON.stringify(result.rows[0]), { EX: 300 });

  return result.rows[0];
}

In this example, the cache expires in:

300 seconds

or five minutes.

The TTL should be set based on:

• update frequency;
• tolerance for slightly stale data;
• volume;
• memory;
• invalidation strategy.

Cache isn't an automatic fix.

A poorly implemented cache can create:

• inconsistent data;
• cache stampede;
• excessive memory usage;
• hot keys;
• operational complexity.

11. Fifth refactor: removing heavy processing from the request path

Consider an API that runs:

receive order
↓
save order
↓
generate PDF
↓
send email
↓
update ERP
↓
log analytics
↓
respond to user

The user ends up waiting on operations that don't need to be part of the response.

We can change this to:

Client
   ↓
orders-api
   ↓
PostgreSQL
   ↓
RabbitMQ
   ↓
HTTP 201

Meanwhile:

RabbitMQ
   ├── pdf-worker
   ├── email-worker
   ├── erp-worker
   └── analytics-worker

Now the request's critical path is much shorter.

The endpoint can go from:

900 ms

to something close to:

150-250 ms

depending on the operation.

12. Production example

Imagine the following environment:

Internet
   ↓
AWS ALB
   ↓
Ingress Controller
   ↓
EKS
   ↓
orders-api
   ├── Redis
   ├── PostgreSQL
   └── RabbitMQ
        ├── notification-worker
        ├── billing-worker
        └── integration-worker

The service initially has:

10 Pods

requests:
CPU:    500m
Memory: 512Mi

limits:
CPU:    1000m
Memory: 1Gi

Each Pod was processing roughly:

100 req/s

before p95 crossed 1 second.

Approximate capacity:

10 × 100 req/s
=
1,000 req/s

After the optimizations:

N+1 removed
cache implemented
indexed queries
parallelized operations
async work moved to RabbitMQ
connection pooling tuned
smaller payloads

Each Pod now processes roughly:

400 req/s

Keeping 10 Pods:

10 × 400
=
4,000 req/s

So a latency reduction can lead to a significant increase in throughput without necessarily quadrupling infrastructure.

13. Kubernetes: application configuration

A production Deployment could use:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
  namespace: production
spec:
  replicas: 4
  revisionHistoryLimit: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1

  selector:
    matchLabels:
      app: orders-api

  template:
    metadata:
      labels:
        app: orders-api

    spec:
      terminationGracePeriodSeconds: 30

      containers:
        - name: orders-api
          image: registry.example.com/orders-api:2.4.0

          ports:
            - name: http
              containerPort: 3000

          env:
            - name: NODE_ENV
              value: production

            - name: PORT
              value: "3000"

            - name: DB_HOST
              valueFrom:
                secretKeyRef:
                  name: orders-api-database
                  key: host

            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: orders-api-database
                  key: password

          resources:
            requests:
              cpu: 500m
              memory: 512Mi

            limits:
              cpu: "1"
              memory: 1Gi

          readinessProbe:
            httpGet:
              path: /health/ready
              port: http
            initialDelaySeconds: 5
            periodSeconds: 5
            timeoutSeconds: 2
            failureThreshold: 3

          livenessProbe:
            httpGet:
              path: /health/live
              port: http
            initialDelaySeconds: 15
            periodSeconds: 10
            timeoutSeconds: 2
            failureThreshold: 3

What this configuration does

The Deployment keeps four instances of the API running and allows a progressive rollout without planned downtime.

The block:

maxUnavailable: 0
maxSurge: 1

allows an extra Pod to be created before an old Pod is removed.

The probes serve different purposes.

Readiness

readinessProbe:

indicates whether the Pod is ready to receive traffic.

Liveness

livenessProbe:

indicates whether the process is still healthy.

The resource values:

requests:
  cpu: 500m
  memory: 512Mi

should be set based on real metrics.

They shouldn't be copied into production without profiling.

14. Autoscaling with HPA

After improving the application, we can use the Horizontal Pod Autoscaler.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: orders-api
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: orders-api

  minReplicas: 4
  maxReplicas: 30

  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60

    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 60

  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65

In this case, the HPA tries to keep roughly:

65% CPU

and can scale between:

4 and 30 Pods

The scale-down stabilizationWindowSeconds helps prevent aggressive scale-down right after a spike.

In production, CPU isn't always the best metric.

For APIs, you can also use metrics like:

requests per second
concurrent requests
latency
pending queue

15. Scaling workers with KEDA

For asynchronous processing, scaling by CPU can be inefficient.

A worker might be using little CPU while thousands of messages wait to be processed.

A better strategy is to use queue depth.

Example with KEDA and RabbitMQ:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: orders-worker
  namespace: production
spec:
  scaleTargetRef:
    name: orders-worker

  pollingInterval: 15
  cooldownPeriod: 120

  minReplicaCount: 2
  maxReplicaCount: 40

  triggers:
    - type: rabbitmq
      metadata:
        protocol: amqp
        queueName: orders
        mode: QueueLength
        value: "50"
        hostFromEnv: RABBITMQ_CONNECTION

In this example, KEDA watches the orders queue.

As the backlog grows, new workers can be added.

The value:

value: "50"

should be calibrated based on:

• average processing time;
• consumer capacity;
• database limits;
• RabbitMQ capacity;
• acceptable queue time.

Scaling consumers without considering the downstream can overload PostgreSQL or external APIs.

16. Connection pooling

Opening database connections has a cost.

Creating a new PostgreSQL connection for every request is a bad strategy.

Use a connection pool.

Example:

const { Pool } = require("pg");

const pool = new Pool({
  host: process.env.DB_HOST,
  port: 5432,
  database: process.env.DB_NAME,
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,

  max: 20,
  min: 2,

  idleTimeoutMillis: 30000,
  connectionTimeoutMillis: 3000,
});

But there's an important detail.

Imagine:

30 Pods
×
20 connections
=
600 connections

If PostgreSQL comfortably supports 300 connections for that workload, the pool is oversized.

So:

pool per application
×
maximum number of Pods

should be part of capacity planning.

In larger environments, tools like PgBouncer can also be used.

17. A timeout is mandatory

Calls without a timeout can hold resources indefinitely.

Example:

const response = await fetch(url, {
  signal: AbortSignal.timeout(2000),
});

Here, the external call has a two-second timeout.

This prevents a slow dependency from tying up:

• a connection;
• memory;
• the event loop;
• a worker;
• a processing slot;

for an indefinite amount of time.

18. Retries: necessary, but dangerous

Retries can improve resilience, but they can also turn a small failure into an avalanche.

Imagine:

10,000 requests

Each request retries three times:

10,000 × 3
=
30,000 additional attempts

If the database is already overloaded, this can drastically worsen the incident.

Prefer:

timeout
+
limited retry
+
exponential backoff
+
jitter
+
circuit breaker

Conceptual example:

100 ms
↓
200 ms
↓
400 ms
↓
failure

Avoid infinite retries.

19. Circuit breaker

If a dependency keeps failing, continuing to send requests can be counterproductive.

A circuit breaker works roughly like this:

CLOSED
↓
failures exceed threshold
↓
OPEN
↓
requests temporarily blocked
↓
HALF-OPEN
↓
test request
↓
CLOSED or OPEN

This approach protects:

• the microservice itself;
• the dependency;
• connections;
• threads;
• the event loop;
• the database;
• network resources.

20. Observability: find out where the 1.2 seconds are going

Without observability, performance tuning becomes trial and error.

A mature strategy uses three pillars.

Metrics

Prometheus can collect:

http_requests_total
http_request_duration_seconds
process_cpu_seconds_total
process_resident_memory_bytes
database_query_duration_seconds
redis_operations_total
rabbitmq_queue_messages

One especially important metric:

histogram_quantile(
  0.95,
  sum(
    rate(
      http_request_duration_seconds_bucket[5m]
    )
  ) by (le)
)

This query approximates the p95 latency using Prometheus histogram metrics.

21. Logs

Logs should provide enough context without exposing sensitive information.

Example:

{
  "level": "info",
  "service": "orders-api",
  "method": "GET",
  "route": "/orders/:id",
  "status": 200,
  "duration_ms": 174,
  "trace_id": "184f4bc...",
  "request_id": "req-78a..."
}

Loki can centralize this data.

Avoid logging:

password
token
full JWT
card numbers
credentials
unnecessary PII

22. Distributed tracing

In microservices, distributed tracing is one of the most useful tools.

With OpenTelemetry we can visualize:

GET /orders/123                  184 ms
│
├── authentication               12 ms
│
├── Redis GET                     3 ms
│
├── PostgreSQL SELECT            41 ms
│
├── inventory-service            82 ms
│   └── PostgreSQL SELECT        38 ms
│
└── serialization                 6 ms

Now we can objectively answer:

Where are the 184 ms going?

Without tracing, the API might look slow when the real problem is actually in another microservice.

23. Alerts

Alerts shouldn't watch availability alone.

A service can be technically UP but practically unusable.

Consider alerts for:

p95 latency > 500 ms
p99 latency > 1 s
HTTP 5xx > 2%
DB pool saturation > 80%
RabbitMQ backlog > threshold
Redis memory > threshold
CPU throttling
OOMKilled
Pod restart rate

These signals are usually more useful than simply:

pod running = true

24. Security

Performance shouldn't come from stripping out security controls.

Secrets

Avoid:

env:
  - name: DB_PASSWORD
    value: super-secret-password

Prefer Kubernetes Secrets integrated with solutions like:

• AWS Secrets Manager;
• External Secrets Operator;
• HashiCorp Vault.

ServiceAccount

Every application should use its own identity.

Avoid sharing administrative credentials.

On AWS EKS, workloads can use Pod-associated identity mechanisms to access AWS services following the principle of least privilege.

NetworkPolicy

When supported by the CNI in use, restrict communication between workloads.

Example:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: orders-api
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: orders-api

  policyTypes:
    - Ingress
    - Egress

  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress

      ports:
        - protocol: TCP
          port: 3000

  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: database

      ports:
        - protocol: TCP
          port: 5432

This policy is just a structural example.

In production, you also need to account for:

• DNS;
• Redis;
• RabbitMQ;
• external APIs;
• observability;
• cross-namespace calls.

An incomplete NetworkPolicy can break the service.

25. FinOps: performance also cuts cost

Performance and FinOps are directly related.

Consider:

Before:

20 Pods
1 vCPU
1 GiB

After:

8 Pods
1 vCPU
1 GiB

If the same demand can be handled with fewer Pods, you can reduce:

• number of nodes;
• provisioned CPU;
• provisioned memory;
• internal traffic;
• database connections;
• compute costs.

On the other hand, poorly sized requests can block consolidation.

Example:

requests:
  cpu: "2"
  memory: 4Gi

if the application uses:

Average CPU: 300m
Average memory: 700Mi

The scheduler reserves far more capacity than needed.

This creates waste.

Requests should account for:

• historical usage;
• p95;
• p99;
• peak behavior;
• overhead;
• operational safety.

The average alone isn't enough.

26. CPU limit and throttling

Consider:

resources:
  requests:
    cpu: 500m
  limits:
    cpu: 500m

If the application temporarily needs more CPU, it can get throttled.

This can produce:

more latency
↓
more concurrent requests
↓
more internal queues
↓
more latency

It's important to track metrics like:

container_cpu_cfs_throttled_seconds_total

CPU limit strategy depends on context and platform policies.

There's no single correct value for every workload.

27. Spot and On-Demand

After improving the application, the infrastructure can also be optimized.

Interruption-tolerant workloads can use Spot.

Example:

Async workers
→ Spot

Critical APIs
→ Spot + On-Demand combination

Database
→ stable capacity

To use Spot correctly, consider:

• multiple instance types;
• multiple zones;
• PodDisruptionBudget;
• graceful shutdown;
• retry;
• queues;
• Pod distribution;
• fallback to On-Demand.

Performance shouldn't depend on a single specific machine type.

28. Development vs. production

Development

It might be acceptable to use:

1 replica
local cache
DEBUG logs
no autoscaling
shared PostgreSQL
simplified timeouts

These conditions make development and testing easier.

Production

You'll usually need to account for:

multiple replicas
PodDisruptionBudget
autoscaling
health checks
timeouts
circuit breaker
connection pool
distributed cache
observability
tracing
NetworkPolicy
Secret Manager
backup
multi-AZ
graceful shutdown
capacity planning

Behavior that works fine at five requests per second locally can fail at thousands of requests per second in production.

29. Load testing

Before claiming the service can handle four times more traffic, you need to test it.

One possible tool is k6.

Example:

import http from "k6/http";
import { check } from "k6";

export const options = {
  stages: [
    { duration: "2m", target: 200 },
    { duration: "5m", target: 500 },
    { duration: "5m", target: 1000 },
    { duration: "5m", target: 2000 },
    { duration: "2m", target: 0 },
  ],

  thresholds: {
    http_req_failed: ["rate<0.01"],
    http_req_duration: ["p(95)<500"],
  },
};

export default function () {
  const response = http.get("https://api.example.com/orders");

  check(response, {
    "status is 200": (r) => r.status === 200,
  });
}

This test progressively increases the number of Virtual Users.

But performance testing needs to reflect real usage patterns.

There's no point testing only:

GET /health

and concluding the system can handle 20,000 requests per second.

You need to reproduce:

• authentication;
• reads;
• writes;
• cache hits;
• cache misses;
• queries;
• messaging;
• real payloads.

30. Before and after comparison

A possible result:

These numbers should be measured in the specific environment.

The main point isn't exactly hitting 180 ms.

The goal is to understand why the service was slow and remove unnecessary work from the critical path.

31. Troubleshooting

When latency starts climbing, analyze it layer by layer.

Check Pods

kubectl get pods -n production -o wide

Look at:

STATUS
RESTARTS
NODE
READY

Check resources

kubectl top pods -n production

Then:

kubectl top nodes

High CPU can indicate saturation.

Continuously growing memory can indicate a leak.

Check the pod description

kubectl describe pod <pod> -n production

Look for:

OOMKilled
Evicted
Unhealthy
FailedScheduling
Probe failed

Check logs

kubectl logs <pod> -n production --since=15m

If there's more than one container:

kubectl logs <pod> \
  -n production \
  -c orders-api \
  --since=15m

Check events

kubectl get events \
  -n production \
  --sort-by=.metadata.creationTimestamp

Check the HPA

kubectl get hpa -n production

And:

kubectl describe hpa orders-api -n production

32. Diagnosing by symptoms

High CPU and high latency

Possible causes:

heavy serialization
loops
encryption
compression
synchronous processing
throttling

Investigate CPU profiling.

Low CPU and high latency

Can indicate I/O waiting:

PostgreSQL
Redis
RabbitMQ
External API
DNS
network

Distributed tracing helps a lot in this scenario.

Too many PostgreSQL connections

Possible causes:

oversized pool
connection leak
too many Pods
slow queries
long transactions

Check:

SELECT
    state,
    COUNT(*)
FROM pg_stat_activity
GROUP BY state;

Database eating high CPU

Check the heaviest queries through the database's observability tools and, when available, pg_stat_statements.

Don't try to fix it right away by just adding CPU.

First figure out which queries are consuming resources.

33. Best practices

A mature performance strategy should follow a similar sequence:

1. measure before optimizing;
2. track p50, p95 and p99;
3. use distributed tracing;
4. reduce synchronous calls;
5. eliminate N+1;
6. build indexes based on real queries;
7. limit payloads;
8. use cache only where there's a clear benefit;
9. move non-critical operations to async processing;
10. configure connection pools deliberately;
11. implement timeouts;
12. apply retry with backoff and jitter;
13. limit concurrency;
14. use autoscaling based on coherent metrics;
15. run load tests;
16. define SLOs;
17. track cost per request;
18. validate performance after each change.

34. What to avoid

Adding Pods without investigating

high latency
↓
add Pods
↓
more connections
↓
more database load
↓
even higher latency

Horizontal scaling doesn't automatically fix downstream bottlenecks.

Indexing every column

Indexes have a cost.

Use execution plans and actual query behavior.

Caching everything

Cache adds complexity and can introduce inconsistency.

Infinite retry

Can cause a retry storm.

Removing CPU limits just to mask problems

Kubernetes resources should be set deliberately, not used to hide an inefficient implementation.

Running slow operations inside the request unnecessarily

Sending emails, generating reports, syncing data and analytics are usually good candidates for async processing.

Benchmarking in an unrealistic environment

A benchmark run without a real database, without authentication, without cache misses and without dependencies doesn't represent production.

35. When to use this approach

This kind of refactoring is especially useful when you have:

high-volume APIs
multi-tenant SaaS
marketplaces
e-commerce
financial platforms
telemetry
event streaming
enterprise integrations
Kubernetes-based processing
microservices architectures

It also makes sense when:

• infrastructure grows faster than traffic;
• adding Pods stopped solving the problem;
• p95 or p99 are elevated;
• PostgreSQL starts saturating;
• many services depend synchronously on each other;
• cost per request is high.

36. When not to use this

Not every application needs Redis, RabbitMQ, a circuit breaker, Kubernetes and dozens of microservices.

If an application handles:

5 req/s

and a 300 ms query doesn't affect any business requirement, introducing:

Redis
RabbitMQ
Kafka
service mesh
KEDA
distributed tracing

can significantly increase complexity without proportional benefit.

Sometimes the best architecture is still:

Application
↓
PostgreSQL

Architecture should match the size of the problem.

37. Practical summary

Going from a latency of roughly 1.2 seconds down to about 180 ms doesn't depend on one specific Kubernetes setting.

It's usually the result of combining several improvements:

Profiling
↓
Better queries
↓
Correct indexes
↓
Fewer round trips
↓
Concurrent operations
↓
Cache
↓
Async processing
↓
Connection pooling
↓
Timeouts
↓
Observability
↓
Autoscaling

The most common mistake is starting with infrastructure.

Adding Pods before understanding the application can just increase cost.

A more effective approach starts by asking:

where are we spending the 1,200 ms?

After instrumentation, we might discover something like:

500 ms PostgreSQL
250 ms External API
180 ms sequential calls
100 ms processing
70 ms other

Now there's a data-driven optimization plan.

If, after the refactor, a request stays active for only 180 ms instead of 1.2 seconds, resources stay occupied for much less time.

This can let the same Pod handle far more requests.

Conclusion

Microservices performance shouldn't be treated purely as an infrastructure problem.

Kubernetes can add Pods. Karpenter can add nodes. HPA can scale Deployments. KEDA can add more workers.

None of these tools, however, automatically fixes a query doing a full scan over millions of rows, an API making 100 queries to build one response, or a microservice sequentially waiting on five external dependencies.

A modern DevOps, SRE and Platform Engineering strategy needs to look at the whole system:

Code
↓
Runtime
↓
Database
↓
Cache
↓
Messaging
↓
Network
↓
Kubernetes
↓
Cloud

Performance should be treated as an architectural property.

First, we measure.

Then, we identify the critical path.

Next, we remove unnecessary processing, cut round trips, optimize queries, parallelize what can be parallelized, move secondary operations to queues, and apply cache where there's a real benefit.

Only after that do we size autoscaling and infrastructure.

The most important outcome of this kind of refactor isn't simply turning 1.2 seconds into 180 ms.

It's turning an architecture that depended on adding infrastructure to survive spikes into a platform that makes better use of every CPU, every connection, every Pod and every available node.

That's the point where application optimization, SRE, Kubernetes, architecture and FinOps stop being separate disciplines and start working toward the same goal: increasing system capacity without proportionally increasing its complexity and cost.