Opening a social app feels simple: refresh the screen and see new posts from people you follow. Behind that interaction, the platform must combine relationships, recent content, privacy rules and ranking without turning every request into an expensive search across the entire database.
The difficulty is not storing a post. It is deciding when to prepare the work that makes that post visible to its audience, then keeping the result useful when people follow new accounts, change permissions, delete content or reconnect after several weeks away.
In this article, we will design a hypothetical Instagram-style news feed. The architecture is an interview example, not a description of Instagram's current implementation.
Introduction
A system design interview might begin with a short prompt: design the home feed for a social application. A useful answer starts by turning that prompt into a set of workflows and guarantees. Otherwise, it is easy to draw a cache, queue and database without explaining what any of them must preserve.
We will begin with a chronological feed of posts from followed accounts. Users can publish text and media references, follow or unfollow another account, refresh the feed and request older results. The design will support private accounts, deleted posts and users who return after a long absence.
Recommendations, advertisements and personalised ranking can be added later. Keeping the initial ordering simple lets us explain the hard parts of distribution, pagination and authorisation before introducing a model that changes which posts deserve to appear first.
The central decision is whether to assemble a feed when it is requested or prepare feed entries when an author publishes. Neither choice is universally best. The right architecture depends on audience size, reader activity, acceptable delay and the cost of rebuilding derived data.
Understand the Requirements and Guarantees
The functional requirements include publishing posts, retrieving an author's recent posts and showing a page of candidates from followed accounts. A post should appear at most once in a returned page, even if several processing paths discover it. Readers should be able to continue without routinely seeing the same entries again.
Privacy has a stronger guarantee than freshness. It may be acceptable for a new public post to take a few seconds to reach a follower. It is a different matter to show private content to someone whose permission has been removed. We therefore distinguish candidate generation from the final decision to return content.
A prepared feed entry means that a post might be relevant to a reader. It does not prove that the reader still follows the author, that the post still exists or that access remains permitted. Those facts can change after the entry is created.
We also need explicit limits. A page might contain twenty posts, while each reader's prepared feed retains only a bounded recent window. Historical browsing can use the authoritative post store rather than requiring every feed to retain every post forever.
For non-functional requirements, discuss response latency, availability, distribution delay and recovery. A hypothetical target could be a fast first page under normal traffic and eventual appearance of new posts within a short, measurable window. Treat these as product assumptions to validate, rather than universal social-platform requirements.
Estimate the Workload
Suppose the application has ten million daily active readers. If each opens ten feed pages a day, the service handles one hundred million page requests daily, or roughly 1,160 per second on average. A tenfold peak would create about 11,600 page requests per second.
Now suppose there are one million new posts per day. That is only around twelve publishes per second on average, which initially suggests a read-heavy workload. However, publishing can trigger many downstream writes. If an ordinary post is distributed to three hundred active followers, it creates three hundred candidate entries.
At those illustrative values, distribution produces three hundred million entries daily before considering retries and exceptionally large audiences. The interactive publish rate does not reveal the true write volume. Capacity planning must count generated work, not just incoming requests.
Read amplification matters too. Returning twenty posts may require loading post summaries, author details, media references and access information. Fetching each item separately could turn one page request into dozens of network or database calls. Batch these lookups and estimate their combined cost.
Record unevenness explicitly. Most authors may have small audiences, while a handful have millions of followers. Readers also differ: someone following fifty accounts creates a different merge workload from someone following twenty thousand. Average audience size hides both extremes.
Separate Authoritative Records from Feed Entries
The authoritative data includes users, follow relationships, posts and access rules. A post record contains its identifier, author, creation time, visibility, media references and current lifecycle state. Media bytes belong in object storage rather than the feed database.
Follow relationships need queries in both directions. Reading the accounts a user follows supports feed generation on demand. Reading an author's followers supports distribution when a post is published. A design that indexes only one direction makes the other unnecessarily expensive.
Meta's historical TAO architecture illustrates why relationship access patterns deserve deliberate storage design. It is useful context, rather than a requirement to reproduce that system in this example.
A feed entry can be much smaller than a post:
viewer_id
post_id
author_id
published_at
distribution_version
The entry stores a reference and ordering information. Loading the current post before returning it avoids distributing a full copy of an author's name, caption and visibility settings into every follower's feed.
Use a unique combination of viewer and post identifiers for each prepared entry. A worker retry can then repeat an insertion without creating another logical feed item. If different feed sources contribute the same post, deduplicate again when assembling the response.
The prepared feed is a derived view. Losing it should be inconvenient and recoverable, rather than equivalent to losing the user's posts or social relationships. That distinction determines which data needs the strongest durability guarantees.
Define the API Contract
A small API surface might look like this:
POST /posts
DELETE /posts/{postId}
POST /users/{userId}/following/{authorId}
DELETE /users/{userId}/following/{authorId}
GET /users/{authorId}/posts?cursor=...
GET /feed?cursor=...&limit=20
The authenticated user determines whose feed is returned. A caller must not obtain someone else's private feed merely by supplying a different viewer identifier. Validate page size and cursor length before performing expensive work.
Publishing should support a stable client operation identifier. If the post is committed but the response is lost, a retry should find the accepted post rather than create another one. This is separate from making distribution workers safe to retry.
The feed response contains post summaries and a continuation token. A summary can include caption text, author information, approved media URLs and counts whose freshness guarantees are documented. Counts need not necessarily be transactionally consistent with every displayed interaction.
A useful response also distinguishes an empty feed from a failed request. Someone following no accounts has no candidates. Someone whose candidate service is temporarily unavailable has an operational problem. Treating both as an empty successful response conceals failures and creates a confusing experience.
A High-Level Design
The architecture separates accepting posts, preparing candidates and serving pages:
Client -> API Gateway -> Post Service -> Post Database + Outbox
|
v
Event Broker
|
v
Follow Graph -> Distribution Workers -> Prepared Feed Store
Client -> Feed Service -> Candidate Merge -> Access Checks
| |
Author Timelines Post Summaries
| |
+-------+-------+
|
v
Feed Response
Media -> Object Storage -> CDN
Post Service
The Post Service validates publishing requests and commits authoritative post state. It should not wait for a million follower-feed updates before acknowledging the author. The author receives confirmation once the post and its durable distribution intent have been committed under the agreed durability policy.
Follow Graph
The Follow Graph provides follower and following lookups, together with the access state required for private relationships. At smaller scale, indexed relational tables may be sufficient. Introduce specialised storage only when measured access patterns justify the additional operational burden.
Distribution Workers
Workers turn accepted post events into candidate entries. They process audiences in bounded batches, record progress and tolerate duplicate messages. Separate interactive publishing capacity from background distribution so a slow audience expansion does not exhaust request threads.
Feed Service
The Feed Service retrieves candidate references, combines them with posts handled on demand, removes duplicates and applies the final access decision. It then hydrates the surviving references into a response. Its latency depends on the entire chain, so each dependency needs a time budget.
Compare Fan-Out on Read and Fan-Out on Write
Fan-Out on Read
With fan-out on read, publishing stores the post in its author's timeline. When a reader requests a page, the service retrieves recent posts from the accounts they follow and merges the results in chronological order.
This avoids preparing feeds for inactive readers. A user who never opens the application does not consume follower-feed storage for every new post. It also makes follow changes relatively natural: the next request can consult the current set of followed accounts.
The difficulty is read amplification. Asking five hundred separate timeline services for data on every refresh is unlikely to produce a predictable tail latency. A slow shard can delay the whole merge, and a reader following many accounts can consume disproportionate capacity.
Batch author lookups where possible, cap the number of candidates retrieved from each source and maintain indexes that support recent posts by author. These measures help, but the system must still perform work related to the number of contributing timelines.
Fan-Out on Write
With fan-out on write, a new post produces a reference in each eligible follower's prepared feed. A reader can then retrieve a small ordered range from one logical feed partition, making the common read path comparatively cheap.
For an author with five hundred followers, this may be an excellent exchange: perform a few hundred background writes once and make repeated reads faster. For an author with ten million followers, one post creates a large distribution job even if only a small fraction of the audience opens the app.
This approach also creates staleness and cleanup obligations. An unfollow, deletion or privacy change does not instantly remove all previously distributed references. The final serving path must account for current state while background cleanup catches up.
Use a Hybrid Policy
A hybrid design distributes ordinary authors' posts in advance while merging very large authors' timelines at read time. It can also restrict precomputation to recently active followers, rebuilding a returning reader's candidates when needed.
Do not define the policy solely as a fixed follower threshold copied from another system. Estimate the write amplification, expected reads, audience activity and storage cost. An account with many dormant followers can differ substantially from one with a smaller but highly active audience.
Store which distribution policy applied to a post or policy version. During a transition, the same post may appear through both prepared and on-demand paths. Deduplication makes overlap safe, while explicit versioning helps diagnose missing candidates.
Make Publishing and Distribution Recoverable
Saving a post and publishing a broker event are two separate operations. If the process crashes between them, the post can exist without ever being distributed. Reversing the calls can announce a post that later fails to commit.
An outbox records the post and its distribution intent in one local transaction. A relay publishes committed intents to the broker. The relay can crash after publication but before recording progress, so consumers must expect repeated events.
Distribution needs two levels of progress. The event identifies the accepted post. A durable job tracks which audience batches have been processed. For a large author, restarting from the first follower after every failure wastes capacity even if duplicate insertions are harmless.
Consider a worker that inserts entries for followers one through one thousand, then crashes before saving its checkpoint. After recovery, it repeats that batch. Unique viewer-post keys prevent duplicate entries, and the worker advances only after the batch's outcome is known.
Audience membership can change during a long job. Decide whether the distribution job uses an audience snapshot or current membership in each batch. Neither choice replaces the serving-time permission check. The important requirement is that changing relationships cannot grant access through a stale candidate reference.
Bound retries and expose failed jobs. A malformed event that fails indefinitely should not block unrelated authors. Quarantine it with enough context to repair and replay, while retaining a record that the post's distribution is incomplete.
Assemble a Feed Page Efficiently
The reader's request starts by obtaining a bounded candidate window. Merge prepared entries with on-demand author timelines, then order the combined candidates using a stable key such as publication time and post identifier.
Hydrate post records and author summaries in batches. Avoid fetching media metadata or counts separately for every row. A request returning twenty items should have a small, predictable number of dependency calls rather than one call per field per item.
Filter deleted, blocked, private or no-longer-relevant candidates before returning them. Filtering may leave fewer than twenty items, so fetch additional candidates within a bounded work budget. Otherwise, a feed containing many invalid references can cause an unbounded scan.
If the budget is exhausted, return the valid items found and a continuation token that advances past the candidates already examined. Do not repeatedly return a cursor that points into the same rejected region. The pagination contract should make partial pages acceptable.
Keep the authoritative access check close to response assembly. For private media, issuing a delivery capability is also an access-sensitive operation. The feed's post text and its media should follow compatible permission and expiry rules.
Design Pagination for Changing Data
Offset pagination asks the service to skip a growing number of items. New entries arriving before the next request can shift positions, producing duplicates or omissions. Deep offsets can also require scanning many earlier candidates.
For a chronological feed, use a cursor representing the last ordering boundary:
published_at < last_published_at
OR
(published_at = last_published_at AND post_id < last_post_id)
The post identifier provides a tie-breaker when several posts share a timestamp. Preserve timestamp precision in the token, and apply the same ordering in every source participating in the merge.
The cursor should be opaque to clients and carry a format version. Bind it to the relevant feed mode and user context, and validate its shape. Encoding a token does not by itself protect it from modification; use an integrity mechanism if altered boundaries would create an unacceptable cost or contract violation.
Ranked feeds need a different discussion. If scores change between requests, the same post can move across the previous boundary. A short-lived feed session can freeze candidate order for one browsing session, while refresh starts a new session. This improves continuity at the cost of session storage and controlled staleness.
Even chronological cursors do not create a historical snapshot of mutable posts. Deletions, access changes and edits must still be applied. Consistent browsing order and current authorisation solve different problems.
Handle Follows, Unfollows and Returning Readers
Following an author should produce a defined experience. The product might backfill a small number of recent posts, show only future posts or rebuild the first page immediately. Without an explicit rule, the user can follow someone successfully and still see no visible change.
Backfill should be bounded. Copying an author's entire history into every new follower's feed can turn a popular account into a source of expensive background work. Reuse viewer-post uniqueness so backfill and normal distribution can overlap safely.
On unfollow, mark the relationship inactive immediately and let the serving path filter the author's candidates. Background removal reduces future scanning cost, but correctness should not depend on completing a large cleanup job before the next refresh.
A returning reader may have no retained prepared feed. Rebuild a recent window from followed authors rather than attempting to replay every post missed during months of inactivity. Show useful recent content first, then populate further candidates asynchronously if the user keeps browsing.
Keep the rebuild path subject to admission and concurrency limits. A notification campaign that brings many dormant users back together can otherwise cause a reconstruction storm against the post and relationship stores.
Cache Candidates Without Making the Cache Authoritative
An ordered cache is useful for recent candidate identifiers. Redis sorted sets provide ordered members and range operations that can support this access pattern. Define equal-score ordering deliberately rather than assuming timestamps are always unique.
The cache should have a bounded retention policy and a rebuild source. Evicting old feed entries is acceptable if older browsing can fall back to a durable representation or author timelines. Evicting the only copy of a post is not.
Use request coalescing when rebuilding a popular reader or author cache entry. Concurrent misses should share bounded work instead of each launching another expensive reconstruction. Add variation to expiry times so large groups of entries do not disappear together after a deployment.
Decide which stale values are acceptable. A slightly stale candidate list may still be useful. A stale permission result can be unsafe. Separate the caching policy for presentation data from the policy for access decisions, and define behaviour when the access service cannot provide a trustworthy answer.
Reliability, Regions and Uneven Traffic
A broker backlog should increase distribution delay before it causes accepted posts to disappear. Monitor the age of the oldest unfinished job and distinguish normal fan-out delay from a job that cannot make progress.
If a large author's post threatens to monopolise workers, split its audience into bounded tasks and allocate capacity fairly across authors. More worker instances help only when storage and relationship lookup capacity can support the extra demand.
Regional caches can make reads faster, but authoritative post and relationship updates need a clear ownership model. Explain where a post is committed, how regional readers learn about it and which access decisions require stronger coordination.
During a regional outage, returning older valid candidates may preserve a useful experience. If the service cannot establish permission for restricted content, it should omit that content rather than interpreting a dependency failure as permission.
Test recovery from a lost cache, duplicate distribution events, a failed audience batch and a regional replica that lags behind a privacy change. Those cases reveal whether the architecture's stated guarantees survive beyond the normal request flow.
Walk Through a Privacy Change During Distribution
Consider an author who publishes a post to an approved private audience. The post transaction commits, and a worker begins distributing candidate references. Halfway through that work, one follower is removed from the authorised audience.
The worker may already have inserted an entry for that reader. It may also be operating from an earlier audience snapshot and insert the entry after the removal. Trying to make every background batch instantly observe every relationship change would add expensive coordination to a large distribution workload.
Instead, the removal changes the authoritative access state. When the reader requests a page, the Feed Service treats the candidate as something to validate. The current decision denies access, so the service discards the reference and continues scanning within its work budget.
This only works if the final check has a defined freshness guarantee. A permission cache that remains stale for an arbitrary period would reintroduce the exposure. The design might invalidate a versioned relationship cache reliably, route sensitive checks to an authoritative owner or omit restricted content when the current permission cannot be established.
Background cleanup can later remove the invalid reference to reduce wasted scanning. It is an efficiency improvement, not the step that makes removal effective. The reader's next request should not depend on a million-entry cleanup completing first.
The same reasoning applies to media. If an earlier response issued a long-lived delivery URL, removing a candidate does not revoke that capability. Choose a media-access lifetime or revocation mechanism consistent with the privacy promise, and explain the boundary between previously downloaded content and future authorised retrieval.
Now consider a distribution worker retrying after the post itself has been deleted. The worker should avoid creating new candidates when it can observe the deletion, but serving-time filtering still protects against races and old entries. The deletion must not disappear merely because an old event is replayed.
Walking through this sequence demonstrates why the feed needs both reliable background processing and a current access decision. One keeps useful candidates available; the other decides which information the product may actually return.
Observability and Validation
Measure feed latency by stage: candidate retrieval, merge, filtering, hydration and response assembly. A single end-to-end timer tells you that the page is slow, but not whether a particular dependency or an unusually large candidate scan caused the delay.
Track distribution lag, duplicate insertions, rejected candidates, empty pages, rebuild requests and work generated by each audience size. Use bounded sampling or aggregation for high-cardinality identifiers instead of putting every user identifier into a monitoring label.
Business-level checks matter. A successful publish response followed by no candidate distribution is a different incident from a feed timeout. Correlate the post identifier through its transaction, outbox event, distribution job and sampled feed outcomes.
Load tests should include hot authors, readers following many accounts, permission changes during distribution and large numbers of returning readers. Uniform random data can make a fragile design look balanced.
Present the Design and Plan Further Improvements
In an interview, walk through one post from author to follower. Explain when the author receives acknowledgement, why the distribution job can be retried and how the reader obtains current content from a lightweight candidate reference.
Then compare the alternatives using the workload. Fan-out on read saves preparation work but increases request cost. Fan-out on write makes common reads cheaper but amplifies publishing. A hybrid policy handles uneven audiences when its transitions and recovery paths are explicit.
Personalised ranking can later sit between candidate generation and final assembly. It should have a bounded input set, measurable latency and a fallback ordering. Recommendations and advertisements can contribute additional candidates without removing the final deduplication and access checks.
Other improvements include better active-reader prediction, adaptive distribution policies and background repair of incomplete feeds. Introduce these only after the basic system has a reliable source of truth and observable recovery behaviour.
Summary
A scalable news feed separates authoritative posts and relationships from the candidate entries prepared for readers. The main architecture decision is when to perform distribution work: on publication, on reading or through a measured combination of both.
Durable publishing, retry-safe distribution and bounded reconstruction keep candidate generation recoverable. Stable ordering, batched hydration and cursor pagination keep the reading path predictable as data changes.
Privacy and deletion checks remain authoritative at serving time. A prepared entry suggests relevance; it cannot grant access after the underlying rules have changed. Making that distinction explicit is what lets the feed stay useful during delays, retries, cache failures and uneven traffic.